Latest

This is Part 2 of our Agent Evaluations series. Here is Part 1 in this series. As AI agents start to gain traction across industries, driving innovation in tasks ranging from customer support to automation of tasks like booking requires their real-world performance evaluation to go beyond static benchmarks. Evaluating

This is Part 1 of our Agent Evaluations series. Here is Part 2 in this series In today’s rapidly advancing world of artificial intelligence (AI), agentic systems are becoming an integral part of numerous industries, powering everything from customer support to robotics. But what exactly are these systems, and

Introduction Unlike deterministic systems, GenAI introduces fundamental unpredictability to production environments. Traditional observability tools fail to capture the semantic drift, hallucination patterns, and prompt-dependent behaviors inherent to large language models. While conventional systems map cleanly from input to output, GenAI requires monitoring the unexplored territory between what we asked for

About Mindtickle Mindtickle is the market-leading revenue enablement platform that combines on-the-job learning and deal execution to drive behavior change and get more revenue per rep. Mindtickle is recognized as a market leader by top industry analysts and is ranked by G2 as the #1 sales onboarding and training product.

Feature spotlight 🔍 Evaluate and monitor AI agents using Maxim It is a constant challenge for AI teams to debug, monitor, and understand what's going wrong in the complex workflows of AI agents, which involve multiple steps, i.e., LLM calls, tool calls, data retrievals, etc. To ensure the

What is RAG? Retrieval-augmented generation (RAG) is a process designed to enhance the output of a large language model (LLM) by incorporating information from an external, authoritative knowledge base. This approach ensures that the responses generated by the LLM are not solely dependent on the model's training data.

The field of web automation faces significant challenges in standardizing agent development and evaluation. BrowserGym, a Gym environment for web automation tasks by ServiceNow, addresses these challenges by providing a unified framework that standardizes the development, testing, and evaluation of web agents. In addition, they also design AgentLab, a complementary

🪪 Customize PII entities based on your needs This change adds a skip list to Maxim's PII evaluator configuration. You can customize which entities you want us to flag as PII. How to customize 1. Click on "Settings" on the PII evaluator page. 2. Select entities you

As we have discussed in our last blog, the o1 family is trained to think before it gives an output to mimic the way humans think for some time before giving an answer to aid reasoning. Beyond the traditional evaluations, we shall go through a few vibe check evals done

Introduction The o1 family of models is trained using reinforcement learning to perform complex reasoning. It is baking in the now familiar chain of thoughts before giving the output. Through the training, the models learn to refine their thinking, explore different strategies, and identify mistakes in those exploration paths. It