Setting up custom token pricing
The custom token pricing structures feature allows you to input your negotiated token costs for more accurate reports. This feature ensures that the total cost displayed in AI evaluations and realtime logs reflects the actual costs incurred by you with special pricing agreements.
How it works
- You provide us with your custom input and output token costs.
- These can be attached to model configs and log repositories.
- Our system uses these rates to calculate the real costs for each evaluation.
- By default, we fallback onto standard pricing.
Setting up custom pricing
To add a pricing structure, navigate to Settings > Models > Pricing tab.
You can then input model name pattern and the cost of your token usage in it. A model pattern must be a string or a regex that matches multiple model names.
Note that cost per 1000 tokens is the only supported unit.
Applying custom pricing to model configs
To attach a the custom token pricing structure to model configs, navigate to Settings > Models > Model configs and edit the model config.
You will see a new section called Pricing structure, here you can select the pricing structure from the dropdown.
You can attach a pricing structure to the following models providers - OpenAI, Azure, Groq, Hugging Face, Together, Google, and Amazon Bedrock.
Applying custom pricing to log repositories
To attach a the custom token pricing structure to a log repository, navigate to Logs in the sidebar and edit the repository.
You will see a new section called Pricing structure, here you can select the pricing structure from the dropdown.
