LLM Pricing

Ujnotes — CC0 1.0

Official pricing hubs, model card registries, and token cost structures for Gemini, OpenAI, Claude, Grok, DeepSeek, and leading frontier AI providers.

What is LLM pricing?

Language model pricing is the metered cost of running inference on large neural networks. Unlike traditional software that sells perpetual licenses or flat monthly seats, large language model APIs charge strictly for compute consumed, measured in tokens—chunks of characters representing roughly three-quarters of an English word.

Every prompt sent to an API consumes input tokens during context processing, and every word, character, or punctuation mark produced by the model consumes output tokens. Because generating new text requires sequential autoregressive decoding steps on GPU tensor cores, output tokens typically cost between two and four times more than input tokens.

Modern provider economics also differentiate between standard real-time calls, cached prompt prefixes, asynchronous batch queues, and hidden reasoning tokens generated during internal chain-of-thought evaluation.

How do pricing structures differ?

Evaluating model costs across different vendors requires understanding five core billing dimensions:

• Input versus output tokens: input tokens cover prompt ingestion, system instructions, few-shot examples, and retrieved context. Output tokens cover generated responses, tool arguments, and structured schema outputs.
• Reasoning tokens: frontier reasoning architectures (such as OpenAI's o-series and DeepSeek-R1) generate extensive internal thinking tokens before producing a visible response. These internal tokens are billed at full output token rates, which significantly increases total query cost even when the final answer is short.
• Prompt and context caching: when multiple requests share identical prompt prefixes—such as static documentation, system instructions, or code repositories—serving engines store precomputed Key-Value (KV) attention states. Reusing cached tokens reduces input costs by 50% to 90% and slashes time-to-first-token latency.
• Context tiering: several frontier models apply higher token rates when context windows exceed standard thresholds (such as 128,000 or 200,000 tokens), reflecting the increased GPU memory bandwidth required to maintain large attention matrices.
• Batch processing discounts: non-urgent workloads submitted to asynchronous batch endpoints (typically with a 24-hour completion window) receive an automatic 50% discount compared to real-time synchronous inference.

Frontier provider pricing and model directories

Frontier AI laboratories maintain dedicated pricing hubs, developer consoles, and model card registries where specifications, benchmarks, and current token rates are published:

• Google DeepMind & Gemini: explore official DeepMind Model Cards for architecture specifications and benchmark evaluations across the Gemini family. Developer rates and context thresholds are detailed on the Google AI Studio Pricing Hub, while enterprise SLAs and provisioned throughput are managed on Google Cloud Vertex AI Pricing. Full model capabilities and context windows are catalogued in the Gemini API Models Documentation.
• OpenAI: token rates for GPT-4o, GPT-4o mini, o1, o1-mini, and o3-mini are published on the official OpenAI API Pricing Page. Context limits, maximum completion tokens, and training snapshot dates are documented in the OpenAI Platform Models Overview. OpenAI provides automatic prompt caching discounts and a 50% discount on Batch API requests.
• Anthropic Claude: rates for Claude 3.5 Sonnet, Claude 3.5 Haiku, and Claude 3 Opus are available on the Anthropic Pricing Hub. Detailed token limits (200k context), 5-minute prompt cache read/write pricing, and Message Batches discounts are outlined in the Claude Models Overview & Rates.
• xAI Grok: developer rates and context specifications for Grok 2 and Grok Vision are indexed on the xAI API Documentation & Pricing. Account billing, team seats, and key limits are managed within the xAI Developer Console.
• DeepSeek: transparent token costs for DeepSeek-V3 and DeepSeek-R1 are published on the DeepSeek API Pricing Page, featuring industry-disrupting base input rates, deep automatic cache hit discounts, and competitive output rates. Architecture details and context configurations are maintained in the DeepSeek API Documentation.

Open-weights and enterprise platforms

Commercial open-weights creators and hyperscale cloud providers offer dedicated managed hosting and model catalogs:

• Mistral AI: commercial API rates for Mistral Large, Mistral Small, Codestral, and Pixtral are available on the Mistral AI Pricing Directory, with technical parameters documented in the Mistral Models Platform Guide.
• Cohere: token pricing for Command R+, Command R, Embed 3, and Rerank 3.5 is listed on Cohere Pricing, with enterprise retrieval and connector guides in the Cohere Models Documentation.
• Meta Llama: open-weight downloads for Llama 3.3 and Llama 3.1 are hosted on the Meta Llama Official Hub. Technical documentation and evaluation benchmarks are detailed in the Meta Model Cards & Prompt Formats. Note that while weights are free to self-host, cloud providers bill per-token fees for managed hosting.
• Amazon Bedrock: unified serverless and provisioned throughput rates for hosting Anthropic, Meta, Mistral, AI21, and Cohere models inside AWS are listed on Amazon Bedrock Pricing.
• Microsoft Azure AI Foundry: enterprise pricing and regional deployment rates for Azure OpenAI and serverless open models are published on Azure AI Services Pricing.

Dynamic price routers and specialized inference engines

When building multi-model routing architectures or optimizing for latency, third-party inference providers and aggregators offer dynamic comparative pricing:

• OpenRouter: maintains a live, normalized comparative index across 300+ models on the OpenRouter Models Directory, tracking real-time token pricing, prompt caching availability, and automated failover routing across providers.
• Groq: provides ultra-fast token inference on custom Language Processing Units (LPUs). Current per-token pricing for Llama 3, Mixtral, and Whisper is published on Groq Pricing.
• Together AI: serverless per-token inference rates and dedicated GPU cluster pricing for open-weights models are indexed on Together AI Pricing.
• Fireworks AI: specialized high-throughput inference rates and fine-tuning pricing for compound AI systems are listed on Fireworks AI Pricing.

How to estimate and manage actual costs

Controlling API expenditure in production systems depends on sound architectural design rather than coupon clipping:

• Route tasks by required capability: avoid dispatching routine classification, data extraction, or summary tasks to expensive flagship models. Reserve flagship reasoning models for multi-step synthesis and route routine workloads to compact models like Gemini Flash, GPT-4o mini, or Claude Haiku.
• Structure prompts for prefix caching: place invariant content—system prompts, tool definitions, schemas, and few-shot examples—at the very beginning of the prompt. Dynamic variables, timestamps, and user inputs should always appear at the end to prevent invalidating the KV cache.
• Leverage batch endpoints for background jobs: evaluations, dataset backfills, embedding generation, and synthetic training runs rarely require sub-second latency. Submitting them to Batch APIs captures a flat 50% discount across major providers.
• Cap reasoning token budgets: reasoning models can generate thousands of hidden tokens if prompt objectives are ambiguous. Set explicit token caps and clear stop criteria to prevent runaway chain-of-thought loops.
• Verify live pricing hubs regularly: model pricing fluctuates as hardware efficiency and competition evolve. Always consult the linked parent directories and developer dashboards before scaling production deployments.

Ai disclosure: written with the help of AI (ChatGPT). You are encouraged to point out errors and omissions.