To learn more about Maxim’s SDK, click here.
Evaluating logs
Auto evaluation
Learn how to use Maxim's distributed tracing solution for your GenAI application.
Maxim now enables seamless import of production logs via our Python and JavaScript SDKs.
Once enabled, all new logs are evaluated in real-time, ensuring actionable insights based on your configured rules.
Maxim’s stateless SDK
Maxim’s stateless SDK simplifies logging by allowing a single configuration across multiple services, nodes, or functions, removing the need for thread pools or manual log sequencing. You can track activities throughout the entire conversation lifecycle with a session object.
- Unified logger configuration: Apply settings across multiple services, nodes, or functions in one step.
- No manual log management: Eliminate the need for thread pools and sequencing.
- Session-based tracking: Monitor activities throughout the entire conversation lifecycle.
Understanding Sessions and Traces
A session provides the overall context for an operation, while traces capture detailed records of activities within that session, offering deeper insights. Maxim also integrates seamlessly with LangChain through the MaximLangChainTracer for easy setup.
- Sessions: Define the context for operations.
- Traces: Capture detailed activity records for in-depth analysis.
- LangChain Integration: Simplify setup using the MaximLangChainTracer.
Refer to the example in the cookbooks to understand the structure.
Setup evaluation for logs
You can also do detailed tracing and debugging with online evaluation:
- Access a trace or session view for comprehensive analysis.
- Set up a continuous evaluation with custom rules:
- Go to the log repository of your choice. On the top right hand side, you will see a button called Configure evaluation, depending on whether you've set up any evaluation configuration or not.
- Once you click on this button, you will see a drop down of options. Click on Edit evaluation configuration to access/set up auto evaluation.
- Specify the sampling rate to determine the percentage of traces to evaluate.
- Apply filters to evaluate only the queries that meet these criteria, e.g., run evaluation only on a range of user feedback
- Select the desired evaluators. These evaluators for online evaluators are only those that do not require ground truth.
- Click the Save Configuration button to begin the evaluation.
After making the evaluation active, all your new logs will start being evaluated based on your sampling rate.