AI Inference Providers Compared: Models, Pricing, Speed, Limits, Privacy, and Availability

Current as of August 17, 2026. Prices are USD unless stated otherwise. This is an independent market analysis, not a sponsored ranking.

Choosing an AI inference provider is no longer a simple model-selection exercise. The same model can arrive through a first-party API, a hyperscaler, a specialized inference cloud, a routing marketplace, a dedicated endpoint, or a container running inside your own network. Those offerings can differ by more than price: they expose different model builds, quantization, rate limits, queueing behavior, retention defaults, failure domains, regional controls, support boundaries, and contractual guarantees.

The most important finding is therefore also the least satisfying: there is no universal “best” inference provider. For interactive applications, a fast specialized host may dominate. For regulated workloads, a region-pinned hyperscaler or self-hosted deployment may be the only admissible choice. For a startup shipping across many models, routing and breadth can matter more than the last few milliseconds. For a mature workload with steady utilization, reserved capacity can beat serverless economics even when its sticker price looks higher.

This report compares 20 concrete, deployable offerings. It also isolates one genuinely comparable lane—OpenAI’s open-weight gpt-oss-120b—and uses independent measurements to show how much the serving layer alone can change performance.

Executive summary

• First-party APIs remain the shortest route to frontier features. OpenAI, Anthropic, and Google generally expose their newest proprietary models, native tools, and model-specific controls first. The trade-off is concentrated vendor dependency and mostly shared-capacity economics unless a premium or provisioned tier is selected.
• Hyperscalers are governance and procurement products as much as inference products. Azure, AWS Bedrock, and Google Cloud combine model access with enterprise identity, private networking, regional controls, consolidated billing, and committed capacity. Their catalogs, prices, regions, and exact deployment modes do not always move in lockstep.
• Specialized inference clouds can radically change same-model latency. In Artificial Analysis’s high-reasoning gpt-oss-120b test, measured output speed ranged from roughly 92 to 1,916 tokens per second across hosts. That result is a serving comparison, not a quality comparison.
• Routing layers buy portability and availability at the cost of another policy boundary. OpenRouter and similar gateways can select among upstreams and fail over, but buyers must evaluate both the router and every eligible upstream. A “zero retention” routing filter is useful, but it does not turn unlike provider contracts into one uniform contract.
• Dedicated and self-hosted deployments shift the problem from API price to utilization. Hugging Face dedicated endpoints and NVIDIA NIM offer isolation and control, but their economics depend on duty cycle, accelerator efficiency, operations, and capacity planning—not token price alone.
• Advertised rate limits are ceilings, not reservations. Shared services may throttle, queue, or shed load below a published maximum. A contractual SLA or provisioned throughput unit is a different product from a high nominal token-per-minute limit.
• Privacy claims are easy to over-compress. “Not used for training,” “zero data retention,” regional storage, and regional processing are four distinct controls. Stateful APIs, abuse monitoring, caches, batch files, grounding, and human-review exceptions can change the answer.
• The defensible buying metric is cost per accepted job. Token price is only one input. Retries, long reasoning traces, tool charges, cache behavior, output length, failure rate, latency, and engineering overhead can reverse a sticker-price ranking.

Scope and inclusion rules

The comparison includes an offering only when a buyer could identify a concrete model or serving configuration, a deployment mode, and a production path on August 17, 2026. The 20 rows deliberately span six archetypes:

1. First-party proprietary model APIs
2. Premium shared-capacity tiers
3. Hyperscaler managed model services
4. Specialized serverless inference clouds
5. Routing and aggregation layers
6. Dedicated or self-hosted infrastructure

The census is representative, not exhaustive. It excludes consumer chat subscriptions, generic GPU rental without a maintained inference product, model catalog entries that cannot be deployed, discontinued endpoints, and preview announcements without a usable access path. It also avoids pretending that every catalog listing is a separate provider: a routed request is attributed to its actual upstream when that upstream is known.

The detailed market matrix

The table below is a readable index. The accompanying 48-field CSV contains the full normalized record for every row, including modality, context, caching, tools, batch support, limits, capacity, SLA, retention, training policy, ZDR, residency, networking, IAM, versioning, procurement, evidence type, caveats, confidence, and benchmark fields.

Scroll or swipe to compare →

What the price column does—and does not—mean

Token prices are normalized to one million input and output tokens where public. They exclude taxes and usually exclude tool calls, retrieval, search, media generation, networking, storage, support, and committed-spend discounts. Reasoning tokens are normally billed as output even when they are not returned. Cache writes and reads often have different prices and minimum lifetimes. Batch and flex prices apply only when the workload accepts their scheduling constraints.

Azure does not expose one stable worldwide number for every deployment combination; its calculator and contract are the authoritative purchasing surface. Dedicated and self-hosted rows are hourly or licensed capacity products, so converting them to $/M tokens without a measured utilization profile would create false precision.

Same model, different serving layer: gpt-oss-120b

Cross-model speed charts are frequently misleading because model size, answer length, reasoning effort, tokenizer, and output quality all affect latency. A cleaner experiment holds the model constant.

The following measurements come from Artificial Analysis’s independent provider benchmark for gpt-oss-120b at high reasoning. Its methodology uses a default 10,000-token input, requests at least 1,500 answer tokens, tests from us-central1-a, and separates time to first token from output generation speed. Figures are snapshots, not guarantees.

Scroll or swipe to compare →

Three conclusions are justified.

First, the serving layer matters enormously: the fastest measured host produced tokens more than 20 times as quickly as the slowest in this captured set. Second, price and speed do not move together. Vertex’s low token price and Cerebras’s very high throughput occupy different attractive corners of the market. Third, a single mean cannot establish tail latency, failure behavior, geographic performance, or sustained capacity. Those require buyer-run tests from the actual deployment regions, at the intended concurrency and prompt distribution.

It would be wrong to mix these numbers with vendor-advertised speed, OpenRouter rolling telemetry, or tests of other models. They use different traffic, clocks, routes, and methodologies.

Provider archetypes and failure domains

First-party model APIs

OpenAI, Anthropic, and Google offer the most direct access to their proprietary models and model-native capabilities. They minimize the lag between a model release and API availability. They also reduce ambiguity about the model developer and primary processor.

Their common weakness is concentration. A first-party outage, policy change, model retirement, or account-control failure can affect every application tied to that API. Premium service tiers reduce capacity risk but do not create provider diversity.

Hyperscaler model platforms

AWS Bedrock, Azure, and Vertex AI align inference with cloud identity, networking, logging, purchasing, and compliance programs. They are often the easiest route through enterprise procurement. They can also provide regional or data-zone deployment modes and provisioned capacity.

The catalog is not uniform. A model may appear later, in fewer regions, with different context limits, or under a different version than the first-party API. “Global” deployment is an availability feature, not a data-locality promise: Azure explicitly allows global deployments to process in any Azure geography.

Specialized inference clouds

Groq, Cerebras, Together, Fireworks, Baseten, and similar providers differentiate through inference hardware, kernels, model optimization, or operating focus. Their appeal is strongest where latency and throughput have direct product value.

The material diligence question is exact serving configuration. “The same model” can still mean different quantization, tensor parallelism, engine, batching policy, context limits, or maximum output. If the configuration is not public, record it as not publicly documented—do not infer it from a benchmark.

Routing marketplaces

A routing layer can standardize an API across many models and upstreams, steer by price or latency, and fail over during provider problems. OpenRouter also exposes controls such as require_parameters and zero-retention filters.

Routing creates a chain, however. The router may avoid retaining prompts while the selected upstream has separate operational logs or regional processing. Automatic rerouting can also change latency and model-provider behavior between requests. Production buyers should pin or allow-list providers when reproducibility, geography, or contract terms matter.

Dedicated endpoints and self-hosting

Dedicated managed endpoints provide isolated replicas and custom serving without requiring the customer to operate the entire stack. Self-hosting with NVIDIA NIM pushes control further: networking, retention, model weights, scaling, and accelerator placement can remain inside the buyer’s environment.

Control is not free. Idle accelerators, cold-start policy, capacity headroom, upgrades, vulnerability management, observability, and on-call ownership become part of the inference bill. Open weights also do not eliminate software support or model-license review.

Shared capacity, provisioned throughput, batch, and flex

These modes solve different scheduling problems:

• Shared standard is elastic and convenient, but capacity is best effort. Published RPM/TPM values describe account limits, not an entitlement to receive those tokens during congestion.
• Premium or priority shared buys preferential scheduling and sometimes a contractual latency/uptime objective. OpenAI Fast mode is unusually explicit: eligible requests receive a 99.9% uptime SLA and a 99%-of-traffic speed objective above 80 output tokens/s.
• Provisioned throughput reserves a capacity unit. Google’s Gemini 3.6 Flash documents 675 adjusted tokens/s per GSU, with output tokens weighted five times. Azure uses provisioned throughput units. Both require workload modeling before comparing cost.
• Batch is typically around half price and targets completion within 24 hours. It is appropriate for evals, enrichment, embeddings, and offline generation, not a user waiting on a screen.
• Flex is cheap best-effort synchronous capacity. Google’s 3.6 Flash Flex Preview targets roughly 1–15 minutes and can reject traffic when capacity is unavailable.

The economic break-even for reserved capacity is:

effective cost per million tokens = total capacity cost / successfully processed tokens × 1,000,000

The denominator must use accepted, non-duplicated work—not theoretical hardware throughput. Include headroom and idle periods.

Latency, reliability, and the metric that matters

User experience depends on at least four clocks:

1. Request admission and queue delay
2. Time to first token or first useful event
3. Output generation speed
4. End-to-end job completion, including tools and retries

Average tokens per second can conceal a poor first-token experience. A good median can conceal painful p95 or p99 queueing. A fast model can still be slow if it produces twice as many reasoning tokens or repeatedly calls a tool.

For production evaluation, calculate:

cost per accepted job = (input + cache + output + tools + retries + routing fees + allocated capacity/ops) / accepted jobs

Then pair it with p50/p95/p99 latency, success rate, rate-limit incidence, output-quality acceptance, and recovery time. This is far more decision-useful than a $/M-token leaderboard.

Reliability also needs a fault tree. First-party API, cloud region, router, upstream, model release, identity system, private link, and customer orchestration can each fail independently—or share hidden dependencies. Multi-provider failover is only real if credentials, quotas, schema compatibility, policy controls, and tests are already in place.

Privacy and data governance

The following statements must not be treated as synonyms:

• Not used for training addresses model or product improvement.
• Retention describes how long content or logs may persist.
• Zero data retention is an eligibility and configuration property, often with feature exceptions.
• Data residency describes storage location; regional processing describes where inference occurs.

Examples illustrate the distinction. OpenAI’s API data controls state that API data is not used for training unless the customer opts in, while default abuse-monitoring logs may be retained for up to 30 days; eligible customers can obtain Modified Abuse Monitoring or ZDR, with stateful and background features requiring separate attention. Anthropic’s retention documentation says Claude Fable 5 has mandatory 30-day retention and is not ZDR eligible. Vertex AI’s ZDR guidance describes abuse-monitoring and grounding exceptions and a project-level in-memory cache that can be disabled.

Groq says inference data is not retained by default, but reliability and abuse logs may persist up to 30 days and stateful features create storage. Together says serverless inputs and outputs are not stored by default and training is opt-in. Cloudflare says customer content is not used to train models without consent; this alone does not establish a universal ZDR or processing-jurisdiction claim.

A routed request inherits the entire chain. Buyers should document: application → router → upstream provider → model developer → tool/search provider → logging/observability destinations. The strictest contractual requirement must be satisfied at every hop.

Model breadth, frontier access, and lifecycle risk

Breadth and depth are different values. A hyperscaler or router may offer hundreds of model identifiers, which helps experimentation and portability. A first-party platform may have fewer families but deeper support for native reasoning, computer use, hosted tools, fine-tuning, or stateful workflows.

“Day one” access should be tested as an operational claim: date available, regions, context, service tiers, quota, and version. A catalog listing without quota or a deployable region is not production access.

Version aliases are another source of drift. An alias can move to a new snapshot, tokenizer, or behavior. Pin dated model versions and container tags when reproducibility matters. Maintain a deprecation calendar, golden evaluation set, rollback path, and schema tests. Anthropic’s Sonnet 5 tokenizer change—approximately 30% more tokens than Sonnet 4.6 by its documentation—shows why a lower per-token price cannot be applied mechanically to historical usage.

Developer experience and operations

OpenAI-compatible request schemas reduce migration effort, but they do not make platforms interchangeable. Reasoning controls, tool-call schemas, streaming events, structured-output guarantees, token accounting, caching directives, error codes, and maximum outputs vary.

A practical portability layer should own:

• provider-specific request translation;
• normalized errors and retry budgets;
• streaming and tool-call event adapters;
• model capability discovery;
• cost and token telemetry;
• policy-aware routing;
• fixture-based contract tests;
• provider and model kill switches.

Do not hide every capability behind a least-common-denominator interface. Keep an escape hatch for model-native features and make the dependency visible.

Procurement checklist

Before signing, ask for evidence—not adjectives—for each of these:

• exact model ID, snapshot, quantization, engine, and context/output limits;
• available regions and actual processing geography;
• rate-limit ceilings versus committed capacity;
• latency and uptime SLO definition, exclusions, measurement point, and credit remedy;
• default retention, abuse review, caches, stateful features, batch files, and tool-provider exceptions;
• training/product-improvement policy and opt-in mechanics;
• ZDR eligibility and feature incompatibilities;
• private networking, IAM/RBAC, SSO, audit logs, encryption, and key management;
• version pinning, deprecation notice, emergency rollback, and end-of-life terms;
• support hours, severity definitions, response targets, escalation route, and financial remedy;
• subprocessor list and change notification;
• export, termination, deletion, and portability provisions.

Conditional recommendations

These are starting hypotheses to validate with a workload-specific bake-off:

Scroll or swipe to compare →

What cannot be concluded from public evidence

This report cannot establish a universal winner, production p99 latency, sustained availability under a buyer’s quota, output quality for an undisclosed workload, or the effective terms of a negotiated enterprise contract. It cannot infer an unpublished quantization or physical processing location from marketing language. It also cannot convert a dedicated hourly endpoint or self-hosted GPU into a truthful token price without utilization, batching, prompt/output distribution, and operational cost.

Public rate-limit tables change by account tier. Dynamic pricing pages and calculators can change without a versioned announcement. Independent benchmark snapshots may test a different region or serving configuration than the one a buyer receives. Those are reasons to run a controlled bake-off, not reasons to ignore public evidence.

A reproducible buyer bake-off

Use a stratified sample of real, de-identified production tasks. Fix model ID, reasoning effort, system prompt, tool availability, maximum output, and acceptance rubric. Run from the real application regions at expected concurrency. Record the raw response, timestamps, usage fields, error, retry, upstream identity, and configuration hash.

Report at minimum:

• quality acceptance rate and human disagreement;
• p50/p95/p99 TTFA and end-to-end latency;
• output tokens per second after first token;
• 429, 5xx, timeout, malformed-output, and tool-failure rates;
• input, cached, reasoning/output, tool, and retry costs;
• cost per accepted job;
• results by prompt-length and output-length bucket;
• results during a sustained-load window, not only isolated calls.

Use paired requests where policies allow, randomize order, warm connections consistently, and publish the configuration. Re-run after model or serving changes.

Machine-readable snapshot

The complete snapshot is supplied as CSV. This compact extract is included for portability:

provider,model_or_configuration,tier,mode,public_price,status
OpenAI,gpt-5.6-sol,Standard,shared,"$5/M input;$30/M output",GA
OpenAI,gpt-5.6-sol,Fast,premium shared,"$10/M input;$60/M output",GA
Anthropic,claude-sonnet-5,Standard,shared,"$2/$10 through 2026-08-31;then $3/$15",GA
Anthropic,claude-fable-5,Standard,shared,$10/$50,GA
Google,gemini-3.6-flash,Standard,shared,$1.50/$7.50,GA
Google,gemini-3.6-flash,Flex,best-effort shared,$0.75/$3.75,Preview
Google Cloud,gemini-3.6-flash,Provisioned,1 GSU,from $1200/week,GA
Microsoft Azure,gpt-5.4 2026-03-05,Global Standard,shared,calculator/contract,GA-registration
Microsoft Azure,gpt-5.4 2026-03-05,Global Provisioned,reserved PTU,calculator/contract,GA-registration
AWS Bedrock,openai.gpt-oss-120b-1:0,Standard,Sydney regional,$0.1545/$0.618,GA
Google Cloud,gpt-oss-120b,MaaS,PAYG,$0.09/$0.36,GA
Groq,openai/gpt-oss-120b,On-demand,serverless,$0.15/$0.60,GA
Cerebras,gpt-oss-120b,Developer PAYG,serverless,$0.35/$0.75,GA
Together AI,openai/gpt-oss-120b,Standard,serverless,$0.15/$0.60,GA
Fireworks AI,gpt-oss-120b,Standard,serverless,$0.15/$0.60,GA
Baseten,openai/gpt-oss-120b,Model API,serverless,$0.10/$0.50,GA
Cloudflare,@cf/openai/gpt-oss-120b,Workers Paid,serverless,$0.35/$0.75,GA
OpenRouter,openai/gpt-oss-120b,PAYG,auto-routed,upstream+5.5%,GA
Hugging Face,openai/gpt-oss-120b on 2x RTX PRO 6000,Dedicated,Ohio replica,$5.50/replica-hour,GA-config
NVIDIA,NIM gpt-oss-120b:2.0.9 MXFP4,NVAIE,self-hosted,"from $4500/GPU/year+infra",GA

Methodology and disclosure

Research was performed on August 17, 2026 using live public pages. Official model cards, API documentation, pricing pages, data-policy pages, and deployment documentation were preferred for product facts. Independent performance numbers are labeled and come only from Artificial Analysis’s same-model provider benchmark. Vendor performance claims were not blended into that table.

Each detailed row identifies the actual provider or upstream, exact model/configuration where public, deployment mode, tier, geography, production status, feature surface, price components, limits, capacity, SLA, data handling, enterprise controls, lifecycle, direct sources, evidence type, caveat, confidence, and verification date. “Not publicly documented” is used where evidence was absent; missing facts were not guessed.

Prices reflect public list prices and exclude negotiated discounts. Policy summaries are technical procurement aids, not legal advice. Readers should verify the signed order form, data-processing agreement, subprocessor list, and regional terms.

The author has no affiliate relationship with the providers in this report and received no payment, free credits, benchmark access, or editorial approval from them. No provider reviewed the conclusions before publication.

Source ledger

First-party and hyperscaler documentation

• OpenAI: API pricing, Fast mode, API data controls
• Anthropic: Sonnet 5, pricing, Fable 5, retention, data residency
• Google: latest Gemini model, Flex inference, Vertex pricing, provisioned model units, Vertex ZDR
• Microsoft: Azure-sold models, deployment types, provisioned throughput, data privacy
• AWS: gpt-oss-120b model card, Bedrock pricing

Specialized, routing, and private deployment documentation

• Groq: pricing, data handling
• Cerebras: public models, rate limits, pricing
• Together AI: pricing, privacy and security, batch
• Fireworks AI: serverless pricing
• Baseten: pricing, regional environments
• Cloudflare: gpt-oss-120b, pricing, data usage
• OpenRouter: pricing, provider routing, ZDR
• Hugging Face: Inference Providers, dedicated endpoint configurator, endpoint access and networking
• NVIDIA: gpt-oss-120b NIM, NIM release notes, NIM licensing

Independent benchmark evidence

• Artificial Analysis: gpt-oss-120b provider comparison, performance benchmarking methodology

Changelog and next review

• 2026-08-17 — Initial edition. Built a 20-offering market census; normalized 48 fields; separated official product facts from independent measurements; added a same-model gpt-oss-120b comparison and conditional recommendations.
• Next scheduled review: 2026-08-24. Check all prices, model status, service tiers, limits, retention/ZDR terms, regions, deprecations, benchmark freshness, and link health. Sonnet 5 introductory pricing deserves special attention before its August 31 expiry.

Continue exploring AI infrastructure

Browse the Kingy.ai AI model database for additional model specifications, or read more independent analysis in the Kingy.ai Blog.

Featured image: AI-generated editorial illustration created for Kingy.ai; it is a conceptual infrastructure visualization, not a photograph or measured benchmark.