Benchmarks

How the cost comparison in Usage & Insights works, and how to read it.

Usage & Insights can compare what you were charged for Aivana MMI with what the same workload would cost at a benchmark's published API pricing. This guide explains what is compared, how to read the estimate, and why some usage is not included. The comparison is an estimate for your information. It does not change your bill.

Frontier Blended is Aivana's recommended reference benchmark. It combines the API pricing of representative leading frontier models into one reference that is not tied to a single provider.

Your Aivana charge is actual. The benchmark amount is an estimate: what the same customer-visible workload, meaning the tokens you were charged for, would cost at the benchmark's pricing. The comparison is made across the whole period, not run by run, so it depends on your mix of requests.

When your total is lower than the estimate, Usage & Insights shows Estimated savings. When it is higher, it shows Estimated cost difference and says how much higher. A higher result is normal for some workloads, for example one with many runs that used live web search. Treat the figure as a guide to relative cost, not as an invoice or a quote for any other service.

• The tokens you were charged for: input, output and the cached part of input.
• Cached input is priced at the benchmark's published cache-read rate, and the rest of the input at its standard input rate.
• Runs that used live web search are included. The separate fees for the search itself are left out, because they cannot be compared fairly across services. See Web Search.

Aivana only compares usage it can price fairly, and never substitutes an invented price. Usage is left out when:

• It happened before benchmark pricing began to be recorded. A price is applied only from the day it took effect, so earlier usage is not compared. Comparisons start from the day benchmark pricing began being recorded, and an account that has been active for longer will see its earliest days left out.
• It was recorded without the information a fair comparison needs.

When only part of the period could be compared, Usage & Insights says so, for example Based on 96.4% of benchmark-eligible usage in this period. In that case the billing shown beside the estimate covers only the compared runs, so the two figures describe the same usage.

A handful of runs says little about cost, so the comparison stays locked until the period has at least 100 comparable runs or 1,000,000 comparable tokens, and at least 90% of that period's usage could be compared. Until then the page shows Build your benchmark profile, with how far along you are, and the rest of Usage & Insights is available as normal. A longer date range may unlock it if you have earlier usage.

Estimates use publicly available API pricing, maintained by Aivana. When a provider changes a price, Aivana adds a new pricing version that applies from its effective date. Each run is measured against the prices in force on the day it ran, so past figures do not change when prices are updated later.

Benchmark comparisons are estimates based on the methodology and pricing information described in this guide. Actual third-party costs may vary. The terms that apply to them are in section 11 of Aivana's Terms of Service.