How does Snowflake handle the cost of Anthropic + OpenAI inference at scale?

Two ways to absorb partner-model cost

There are only two structural answers to the question of who eats the token bill, and Snowflake runs both at once.

The first is passthrough resale. Snowflake contracts with Anthropic and OpenAI (and, for some Claude deployments, reaches them through AWS Bedrock in the same cloud region as the customer's account), buys inference in bulk, and re-meters it to customers as Cortex credit consumption. The customer never sees a per-token invoice from a model vendor; they see a line of credits on a Snowflake bill, converted at whatever rate their edition and region carry. Snowflake's published Service Consumption Table maps each supported model to a credits-per-million-tokens figure, so the conversion is transparent even though the underlying wholesale rate is not. Economically this is a distribution business: Snowflake earns the spread between what it pays the model provider and what the credit conversion charges, plus the serverless compute burned running the function itself. The spread is real but structurally thin — nothing like the margin on storing and querying a petabyte, because the dominant input cost is someone else's GPU time, not Snowflake's.

The second is first-party inference. Snowflake's own Arctic and Arctic-Embed models run on infrastructure Snowflake already owns and already knows how to amortize. There is no per-token payment leaving the company. Every embedding call, every simple classification, every SQL-generation request that lands on an in-house model instead of Claude Opus or GPT-class reasoning converts a passthrough transaction into a compute transaction — the shape of business Snowflake's core product gross margin, publicly reported in the mid-70s percent range, was built on. That is why Arctic exists as a product strategy and not just a research artifact.

Customers face the mirror image of the same choice: buy inference through Cortex, or go direct to the Anthropic and OpenAI APIs. Going direct gets you the raw list rate with no intermediary spread, day-one access to new model versions, and full control over caching, batching, and streaming parameters. What it costs you is everything Cortex exists to provide. Direct means you move governed data out of the security perimeter it already lives in, stand up and maintain a pipeline to do it, re-implement row-level security and masking on the way out and the way back, keep a second set of credentials and rate limits, and reconcile a second vendor invoice against the workload that produced it. Cortex means the data never leaves the account boundary, the prompt inherits the RBAC and masking policies already attached to the table, the spend lands on the contract you already negotiated and the commitment you already drew down, and the model call is a SQL function any analyst can write.

The honest framing for a buyer is that Cortex is not competing to be the cheapest source of raw tokens and will not win that fight. It competes on the total cost of a governed inference pipeline — including the engineering headcount you do not spend building one, and the compliance conversation you do not have to reopen because customer PII never traversed an egress path. For a small, single-purpose, high-volume application with no sensitive data, direct APIs usually win on cost. For anything that reads governed warehouse tables, the calculus flips fast.

How to decide which path a workload belongs on

Do not make this decision once, at the platform level. Make it per workload, because the correct answer genuinely differs across the workloads inside a single account.

Four questions decide it. Does the prompt read governed data? If the input is assembled from tables carrying masking policies, row access policies, or contractual data-residency terms, the default is Cortex — the cost of rebuilding those controls outside the perimeter dwarfs any token spread. How hard is the task? Extraction, classification, routing, sentiment, and short summarization are solved problems for small models; only genuinely open-ended synthesis, long-context reasoning, and multi-step planning justify frontier-tier pricing. How repetitive is the input? A workload that re-sends the same 40,000-token corpus on every call is a caching problem before it is a model-selection problem. Is it interactive? Anything that can tolerate minutes of latency — nightly enrichment, backfills, bulk scoring — belongs on a batch path, where Anthropic and OpenAI both offer roughly half-price asynchronous processing.

The failure mode is picking one frontier model, wiring it to every function in the account, and discovering the bill ninety days later. In practice the first Cortex bill that alarms a finance team is almost never caused by the model being expensive; it is caused by the wrong model doing cheap work, uncached, at volume, inside a loop nobody profiled.

The numbers behind each path

Concrete arithmetic makes the spread obvious. Published list pricing as of mid-2026: Claude Opus-class models run roughly $15 per million input tokens and $75 per million output tokens; Sonnet-class sits near $3 input and $15 output; Haiku-class lands near $1 input and $5 output. OpenAI's frontier and reasoning tiers occupy comparable bands, with a mini tier serving as the cheap workhorse. Always check current vendor pricing pages before modeling — these tiers move.

Take one million tokens through an Opus-class model. Split evenly, 500,000 input and 500,000 output, the raw cost is 0.5 × $15 plus 0.5 × $75 — $7.50 plus $37.50, about $45. A more realistic retrieval-augmented mix of 800,000 input and 200,000 output costs 0.8 × $15 plus 0.2 × $75 — $12 plus $15, about $27. Output tokens are five times the price of input tokens in that tier, which is the single most important pricing fact in the entire stack and the one most teams ignore. Truncating verbose model output is often a bigger lever than trimming context.

Now run the identical million through a Haiku-class model at the same 800/200 mix: 0.8 × $1 plus 0.2 × $5 — $0.80 plus $1.00, about $1.80. That is roughly a fifteen-fold difference in raw cost for the same token volume. On extraction and classification work, where the small model's answer is indistinguishable from the frontier model's, that ratio is free money left on the table.

Layer on the multipliers that turn a modest per-call cost into a six-figure quarter. Retrieval width: moving from top-5 to top-50 chunk retrieval can inflate input tokens by an order of magnitude, and input is where RAG spends. Agent loops: a Cortex Agent that chains eight to fifteen model calls per user request multiplies the bill linearly — one agent question can cost what twenty chat completions cost. Extended reasoning: thinking tokens bill as output, at output rates, so a single hard question in a reasoning tier can cost several times a normal completion.

Then the discounts. Anthropic's prompt caching reduces the cost of cached input reads by roughly 90 percent, with a smaller premium charged on the initial cache write; for any workload where a large fixed corpus or system prompt repeats across calls, this is the largest single lever available and it requires no model change. Batch processing on both vendors runs around 50 percent off synchronous rates for latency-tolerant work. Stack caching on a repetitive RAG workload and batching on the nightly portion, and a bill can fall by half before anyone changes a model.

On the Snowflake side, credit conversion determines what the customer actually pays — and since April 2026 that no longer depends on your edition. Snowflake now bills AI work in AI Credits, a currency separate from the Platform Credits that cover warehouses and storage, and an AI Credit costs the same on Standard, Enterprise, Business Critical and VPS: $2.00 per credit with global routing, $2.20 when routing is pinned to a region for residency or compliance. AI Functions, Cortex Agents, Cortex Search and Cortex Code all draw on that pool. Two accounts on different editions running byte-identical AI workloads now generate the same AI Credit charge; the edition spread that used to pull those invoices apart survives only on Platform Credits, so it still moves the cost of the pipeline around the model rather than the inference itself. The serverless compute that orchestrates the call, performs the retrieval, and assembles the result burns credits too. Per call it is small; across a loop that fires fifteen times per user question, it is not.

The net picture: Snowflake's take on partner-model inference is a modest percentage of the line item, with the majority of the dollars flowing to the model provider and, on Bedrock-routed calls, to AWS. That is a distribution margin, not a software margin — which is exactly why the routing layer and Arctic matter so much to the company, and why Snowflake's own cost-optimization guidance points customers toward cheaper models and caching rather than away from them. The interests genuinely align here: volume that shifts to first-party inference is better for Snowflake's margin *and* cheaper for the customer.

Implementing the split, in sequence

Order matters. Teams that jump straight to swapping models without instrumentation end up guessing, and teams that instrument without an eval harness end up shipping quality regressions they cannot detect.

Instrument first. Before changing anything, get per-workload visibility: tag every Cortex call path by application, attribute credit consumption in the account usage views, and produce a two-week baseline of cost by function, by model, and by workload. You want to know which three workloads generate seventy percent of spend, because that is where every subsequent change should land.

Build the eval set second. Assemble 100 to 300 representative inputs per workload with known-good outputs or a scoring rubric. Without this, a model downgrade is a leap of faith and any regression surfaces as a user complaint weeks later. With it, tiering becomes a measurement rather than an argument.

Tier third. Move classification, extraction, routing, and embedding to Arctic or the cheapest tier that clears your eval bar. Keep frontier models on the genuinely hard tail. Expect the tiering pass alone to remove a large fraction of token spend, because the cheap-work-on-expensive-models pattern is nearly universal in first-generation deployments.

Cache fourth. Identify the fixed prefix in each prompt — system instructions, schema descriptions, static reference corpora — and structure prompts so that stable content sits at the front where it can be cached and volatile content at the back. Layer a result cache above it for genuinely repeated questions.

Trim the retrieval fifth. Tune top-k downward until eval scores start to move, then step back once. Most RAG pipelines ship with a k chosen by intuition and never revisited, and each unnecessary chunk is paid for on every single call forever.

Then govern. Attach Snowflake budgets to the objects backing AI workloads so serverless spend triggers alerts before it triggers a board conversation, set separate thresholds for dev and production, and route the alerts to the team that owns the workload rather than to a shared inbox nobody reads.

Renegotiate last, once you have three to six months of real consumption data. Commitment sizing is only as good as the forecast under it, and a forecast built on the pre-optimization bill locks you into a commitment you will struggle to burn down.

What RevOps should ask for before the next renewal

RevOps teams are usually the ones who inherit this bill, because the workloads that consume the most inference — account summarization, call transcript analysis, lead enrichment, next-best-action scoring, natural-language reporting over the warehouse — are the ones RevOps commissioned. Three asks make the difference between a defensible line item and an unexplainable one.

Ask for a unit metric, not a total. "Cortex cost $X last month" is unmanageable. "Cost per enriched account is $0.04, cost per summarized call is $0.11, cost per analyst question is $0.02" is manageable, forecastable, and comparable against the manual process it replaced. Insist that every AI workload publishes its denominator alongside its spend. When a unit cost triples, you find out in a week instead of a quarter.

Ask for the model-mix report. A monthly breakdown of tokens by model tier answers the only question that matters at review time: how much frontier-tier spend is doing work a cheap tier could do? If eighty percent of your tokens are hitting a frontier model, the optimization is sitting right there. If ten percent are, you have already done the work and the remaining lever is caching or commitment pricing, not tiering.

Ask what happens when the vendor changes price. Model pricing has moved substantially and repeatedly, generally downward per unit of capability. Build the forecast so a rate change is a parameter, not a rebuild, and make sure the team owning each workload knows whether a cheaper successor model has shipped that clears the eval bar. Re-running the eval harness against a new model tier is a half-day of work that has repeatedly paid for itself.

The strategic read is straightforward: Snowflake does not handle Anthropic and OpenAI inference cost by absorbing it, and no platform in this category can. It handles it by aggregating buying power, metering consumption through credits, and steering as much volume as quality allows onto cheaper tiers and its own models. Customers who understand that structure get to ride the same levers — tiering, caching, batching, retrieval discipline — and reach a bill that behaves like infrastructure spend instead of a variable tax on curiosity.

Related questions

Is Cortex more expensive than calling Anthropic or OpenAI directly?

On raw tokens, generally yes — Snowflake earns a spread. On total delivered cost, often no, once you price the egress pipeline, duplicated access controls, second vendor contract, and engineering time that a direct integration on governed data requires.

Which Cortex workloads waste the most money?

Retrieval-augmented pipelines with oversized top-k, agent loops that chain many calls per user request, verbose outputs billed at output rates, and cheap tasks like classification and extraction left running on frontier-tier models.

Does prompt caching work through Snowflake Cortex?

Caching is the largest non-model lever available, cutting repeat input cost dramatically for workloads with a stable prefix. Check current Cortex documentation for which caching behaviors are exposed on your models and region before assuming it is on.

How do I stop a runaway Cortex bill mid-month?

Attach Snowflake budgets to the objects backing AI workloads with alert thresholds well below your ceiling, split dev and production limits, and route alerts to the workload owner. Detection speed matters more than a hard cap.

Should we standardize on one model across the account?

No. Standardize on one *interface* and one eval harness, then let each workload sit on the cheapest tier that clears its quality bar. Single-model standardization is the most common source of avoidable inference spend.

FAQ

Where does Snowflake's margin on partner-model inference actually come from?

From the spread between negotiated wholesale rates and the credit conversion charged to customers, plus the serverless compute that runs orchestration, retrieval, and governance on each call. It is a distribution margin on someone else's GPU time, structurally thinner than the margin on Snowflake's core storage and compute business, which is why volume and routing matter so much to the economics.

Why does Snowflake push customers toward cheaper models if it earns a spread on expensive ones?

Because a shift to first-party inference converts a passthrough transaction into a compute transaction on infrastructure Snowflake already owns, and because surprise bills produce churn and billing disputes rather than durable consumption. Optimization advice that lowers a customer's cost per unit of work typically increases the number of workloads that customer is willing to run.

Does routing Claude models through AWS Bedrock change what I pay?

Snowflake does not itemize a separate Bedrock fee; it is bundled into the credit charge for the call. The customer-visible cost is the credits-per-million-tokens figure in the Service Consumption Table multiplied by the credit price for their edition and region, regardless of the routing path underneath.

How much can tiering and caching realistically save?

The ratio between a frontier tier and a small tier is roughly an order of magnitude on identical token volume, and cached input reads cost a small fraction of uncached ones. Real accounts that instrument, tier, cache, and trim retrieval commonly cut spend substantially without a measurable quality change, because the initial deployment was almost never optimized.

What is the right first move on a Cortex bill that just tripled?

Do not change models yet. Pull consumption by workload and model, find the two or three paths generating most of the spend, and check three things in order: is a cheap task running on an expensive model, is a repeated prefix going uncached, and is a loop firing more calls per request than anyone intended. The cause is usually one of those three.

Do credit prices differ enough between editions to matter?

Not for the AI itself, not since April 2026. Snowflake bills AI services in AI Credits, priced identically on Standard, Enterprise, Business Critical and VPS — $2.00 per credit on global routing, $2.20 when you restrict routing to a region. Edition still moves your Platform Credit rate, so it changes what the warehouses, storage and serverless compute around the model cost, not the inference. When you model inference cost, model it in credits first, then apply your own contracted credit rate — never assume a published dollar example matches your account.

Sources

• https://docs.snowflake.com/en/user-guide/snowflake-cortex/llm-functions
• https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf
• https://docs.snowflake.com/en/user-guide/snowflake-cortex/pricing
• https://docs.snowflake.com/en/release-notes/2026/other/2026-04-06-ai-services-billing-breakout
• https://docs.snowflake.com/en/user-guide/budgets
• https://www.anthropic.com/pricing
• https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
• https://openai.com/api/pricing/
• https://aws.amazon.com/bedrock/pricing/
• https://github.com/Snowflake-Labs/snowflake-arctic
• https://investors.snowflake.com/

Related on PULSE

• [How does Salesforce handle the cost of OpenAI plus Anthropic API spend at scale?](/knowledge/q1557)
• [How do AI inference costs and AI product gross margins work in 2027?](/knowledge/q13059)
• [How do you optimize LLM inference cost in production in 2027?](/knowledge/q12293)
• [Why are 2027 B2B buyers demanding AI-generated demo personalization at scale?](/knowledge/q16516)
• [Why are GTM leaders rethinking account-based strategies as AI personalizes outreach at scale in 2027?](/knowledge/q16247)