AI Economics Series Article #6 | The consumption layer: where enterprise AI economics will be won or lost

As model choice widens, the decisive enterprise capability will be the control plane that routes demand across models, data, tools and hosting environments, and proves that each choice makes economic sense.

The first generation of enterprise AI strategy was largely about access. Companies wanted an approved route to a powerful model, preferably through a cloud provider and wrapped with security controls. At the time, that felt like progress. In many organisations, it was.

Access is no longer the hard part. The market now offers frontier models, smaller proprietary models, specialist systems and downloadable open-weight alternatives. Google Cloud’s Model Garden alone advertises more than 200 available models.[1] What looked like a model-selection problem two years ago is turning into a portfolio-management problem.

That shift changes the economics. Each source of machine intelligence comes with its own combination of quality, price, latency, data treatment, operational resilience and regulatory exposure. Add retrieval services, licensed data, external tools and agents, and what the enterprise is buying starts resembling a supply chain for intelligence.

The consumption layer sits between that supply and the business demand for it. Done well, it decides where work should run, which data it may use, what it may cost and how much control the task requires. Otherwise it ends up as an API gateway sitting in front of a growing set of invoices.

 Why this layer matters

The term consumption layer sounds technical, almost administrative. It is more important than that. A conventional gateway accepts traffic and sends it somewhere. An enterprise AI consumption layer has to make judgements: who is asking, what kind of task this is, whether the request touches personal or regulated data, whether the work is routine or consequential, whether it must stay within a particular country or data zone, how much latency is tolerable, and what completing it may reasonably cost.

Those decisions used to be made during system design. A team chose a model, embedded it in the application and revisited the choice only when something broke or a contract came up for renewal. That approach is becoming untenable. Models, prices and deployment options change too quickly for that, workloads are too mixed, and agents can invoke several services during what looks to the user like a single task.

Without a common control layer, each product team invents its own answer. One team uses the strongest model by default because it produced the best demonstration. Another writes its own routing logic, and a third self-hosts for reasons of sovereignty without testing whether utilisation will support the fixed cost. The architecture fragments. So do the controls and the bills.

The enterprise may still describe the result as a platform. Economically, it is a collection of local choices that nobody can see in full.

Model choice is moving into runtime

The simplest way to waste money on AI is to treat every request as equally difficult. An enterprise workflow rarely contains one kind of work. A customer-service process might classify an enquiry, retrieve account information, interpret policy, draft a response and decide whether a human must intervene. The classification is routine. The policy judgement may not be. Sending every step to the same premium model is convenient, but it prices the whole workflow as though every step were hard.

Model routing offers a better approach. Microsoft now exposes balanced, cost and quality routing modes, while Amazon Bedrock offers intelligent prompt routing within model families. AWS says its routing can cut model costs by up to 30 per cent without reducing accuracy in its tests.[2][3] That is a vendor-reported figure from the vendor’s own tests, and it will not transfer intact to every estate. The broader point still holds. Capability can be matched to the task instead of bought indiscriminately.

There is a catch. Routing is only as good as the evidence beneath it. Public benchmarks are useful, but they do not tell a bank which model handles its credit language most reliably, or a manufacturer which one interprets its maintenance records with the fewest costly mistakes. Enterprises need their own evaluations, built from their own documents, edge cases and control requirements.

Once that evidence exists, model selection stops being a matter of taste. A premium model can be justified where it materially reduces error, review effort or risk. A smaller model can take the high-volume work where the quality difference is immaterial. Allegiance to a provider gives way to an operating decision that can be revisited when the evidence moves.

The real unit is the completed outcome

Public price cards make AI look deceptively tidy. Input tokens cost one amount, output tokens another. Cached tokens are cheaper. Batch processing may attract a discount. Tools such as search, file retrieval and code execution may be charged separately.[5] It is easy to compare the visible rates and conclude that the lowest price is the best deal. Production systems refuse to stay that simple.

A cheaper model may need a longer prompt, more examples, repeated retrieval or several attempts before it produces an acceptable answer. It may shift work into human review. A more expensive model might complete the task in one pass. Self-hosting may remove an external API bill, but it brings GPU capacity, platform engineering, monitoring, patching and security into the cost base.

The FinOps Foundation calls this measure “use case economics” and defines it as “the total cost of achieving a specific business outcome, measured per unit of that outcome.”[4] That is a more serious measure than token spend. It asks what the enterprise paid to resolve a customer query, process a document, test a control or complete a code review, counting model inference, retrieval, data services, tools, orchestration, hosting, monitoring, human review and the expected cost of rework or failure. The lowest-priced model will not always produce the lowest-cost reliable outcome.

This is where AI economics turns from metering into management. Vendors sell tokens, capacity and calls. The enterprise has to account for work completed.

 Three routes to intelligence

A mature consumption layer does not force the organisation into a single philosophy about models. It gives the enterprise several routes and allows each workload to earn its place on one of them.

The frontier route is for work where better reasoning genuinely changes the result: complex research, difficult software engineering, ambiguous document interpretation, multimodal analysis or an agent that must plan across several systems. Premium intelligence belongs on premium problems. Using it for routine extraction is the AI equivalent of putting every passenger in first class.

Most bounded enterprise work belongs on the efficiency route. Classification, summarisation, translation, drafting and straightforward retrieval usually need dependable performance more than frontier capability, and smaller proprietary models and specialist systems can offer lower latency and better unit costs for them.

Then there is the controlled route, built on open-weight or privately deployed models, for cases where data sensitivity, customisation, resilience or sustained volume alters the calculation. Meta makes Llama weights available for download and says the models can be customised and run on premises, in the cloud or locally without sharing data with Meta.[6] Mistral has released models under the Apache 2.0 licence and supports custom and private deployment options.[7] These choices give enterprises more control over the inference environment and over the pace at which they adopt changes.

None of this creates free intelligence. Open weights change part of the licensing equation. Compute, engineering, security and maintenance all remain. Managed services may look expensive per unit and still be cheaper overall when demand is volatile or internal engineering capacity is scarce. Self-hosting becomes attractive when volume is high, utilisation is strong and the organisation is prepared to run the service properly.[4]

The consumption layer lets all three routes coexist. The argument worth having is rarely frontier versus open weight. It is which path gives this workload the right balance of capability, cost and control.

Sovereignty costs money, but so does dependency

Sovereign AI is often reduced to a question about where the data centre sits. Location matters, but it is only the beginning. Microsoft describes digital sovereignty through data controls, operational controls and technological independence.[8] That framing is useful because a workload can store data in one jurisdiction while relying on an external control plane, foreign operating staff or a proprietary service that cannot be replaced quickly. Residency alone does not answer those questions.

Different workloads need different levels of control. Public-information summarisation may be suitable for a globally delivered service. A customer-data process is more likely to need a regional or data-zone deployment, and critical public-sector or defence workloads can require dedicated, disconnected or air-gapped infrastructure. Google Cloud’s sovereign offering explicitly spans public-cloud data boundaries and connected or air-gapped deployments.[9] Microsoft Foundry likewise distinguishes global, data-zone and regional processing, as well as pay-per-token and reserved-capacity deployment types.[12]

Each step towards tighter control has an economic consequence. Regional processing can narrow capacity options, and dedicated environments require commitments. Air-gapped systems give up some of the convenience and pace of public-cloud services, while private hosting moves operational responsibility back to the enterprise.

The sensible objective is enough sovereignty for the task, priced honestly. Maximum sovereignty everywhere is not the goal. Buying more control than a low-risk workload needs is wasteful, and ignoring dependency in a critical workload is another way of underpricing risk.

Agents make hidden consumption visible - eventually

Agents sharpen the problem because they conceal the path between an instruction and an outcome. A user may ask for one thing: investigate a complaint, prepare a client briefing, reconcile an exception. The agent may classify the request, retrieve documents, call two models, search an external source, query a system of record, run code, ask another model to check the answer and then retry when one step fails. The interface shows one task. The bill records everything the agent did to get there.

That chain is variable. A clean case finishes quickly. An ambiguous one can loop through extra searches and tool calls before it escalates to a person, and average cost then looks harmless while a small number of expensive runs distort the economics.

Task-level budgets are therefore as important as application budgets. An agent needs limits on tokens, tool calls, retries and elapsed time, rules about which models it may call and which systems it may change, and a point beyond which further machine effort is worth less than human judgement.

Modern AI gateways are beginning to provide the underlying mechanisms: token quotas, semantic caching, load balancing, health checks and circuit breaking.[10][11] Those controls are usually presented as resilience or platform features. They are also margin controls. An agent that cannot be stopped, rerouted or capped is an open-ended liability disguised as automation.

The model is only part of the supply chain

The phrase AI consumption encourages a narrow focus on models. Real enterprise systems consume much more. A research agent may call a foundation model, enterprise search, a vector database, licensed market data, document stores and a code-execution environment. A service agent may use identity, CRM, product, payment and communication systems. Some components are priced by token, others by query, licence, transaction, storage or compute time. Human review sits alongside them all, often without appearing on the technology invoice.

The consumption layer should therefore govern four kinds of supply: models, data, tools and human judgement. That broader remit creates an important discipline. Not every problem needs a language model. A database query, a rules engine or a conventional analytic model is often cheaper, faster and more reliable. A good control plane exists partly to keep AI out of the places that do not need it.

Consumption data becomes an enterprise asset

The most valuable output of the consumption layer may turn out to be the evidence it accumulates rather than any single routing decision. The platform can show which tasks really need frontier reasoning, where long context improves results and where it merely adds cost, which agents retry too often, which data sources improve accuracy and which teams generate activity without corresponding business value. That knowledge can inform architecture, procurement and product design.

It can also change the make-or-buy decision. A workload that begins as occasional API consumption may eventually reach the scale and stability that justify reserved capacity or private hosting. Another looks large until caching and prompt redesign remove half the demand. And a heavily used feature can deserve to be withdrawn because users like it without it changing any meaningful outcome.

Over time, routing policy becomes proprietary operating knowledge. Competitors may have access to the same models. They will not automatically know which combination of model, data, tools and control produces the best economics inside this organisation.

Avoiding the central-platform trap

There is an obvious danger in all of this. Faced with fragmented consumption, the enterprise may build a central platform that controls everything and enables very little. If adding a model or tool requires months of review, product teams will work around the platform. Shadow AI will replace shadow cloud. Economic control cannot depend on a committee examining every prompt.

The better design borrows a concept from platform engineering: a set of paved roads, the term Netflix's engineering teams popularised for routes that guide developers without gating them. The centre supplies identity, approved model catalogues, common evaluations, telemetry, data-boundary rules, cost allocation and minimum controls. Product teams retain responsibility for the business outcome, domain-specific testing and the user experience. Low-risk work moves quickly through pre-approved routes. Consequential work carries stronger evidence and tighter constraints.

Dynamic routing itself needs boundaries. Microsoft notes that automatic updates to a router’s underlying model set can affect both cost and performance.[3] That may be acceptable for low-risk assistance. It is harder to defend when a regulated process needs reproducibility. In those cases, the route, model and version may need to be fixed until a controlled change is approved. Control here means knowing where variation is safe, which is not the same as insisting on uniformity.

Who owns the economics?

No technology team can settle these trade-offs alone. The business owner knows what a successful outcome is worth. Technology understands service design and operational efficiency, while risk, legal and data functions set the permitted boundaries. Procurement manages supplier leverage and concentration, and finance and FinOps bring allocation, forecasting and benefits discipline.

The problem starts when each function optimises its own slice. Technology lowers token cost. Risk adds review. Procurement secures a discount. The business drives adoption. Nobody can say whether the cost per outcome has improved.

Every production AI service should have three named accountabilities: a business owner for value, a technology owner for service and consumption, and a control owner for permitted use. They should work from the same evidence, not from separate dashboards and separate definitions of success. That shared view is what turns a consumption layer into an economic management system.

What boards should ask

Boards do not need to approve individual model routes. They do need confidence that management has control over aggregate demand. Can AI consumption be traced to a business process rather than only to a vendor? Does the organisation know which work genuinely requires frontier capability, and whether sensitive tasks can be directed to regional, sovereign or privately controlled environments? Are agents operating within task-level limits, with cost reported per completed outcome? And has management designed portability, or merely placed a common interface over a dependency it cannot unwind?

The most revealing question is a simpler one. Where is rising AI consumption producing rising value, and where is it not? A company that cannot answer is still experimenting, however sophisticated its platform may look.

Economic control is the advantage

The AI market will remain unsettled. Models will improve, price structures will change, and suppliers will keep rearranging the boundary between their platforms and the enterprise. Agents will make more decisions about which services to consume, often without a person watching each step.

Enterprises cannot manage that volatility by repeating a procurement exercise every time the leaderboard changes. They need an enduring layer that can match the level of intelligence to the work, apply the right degree of control and show what the choice cost.

The winners will not necessarily be the companies that standardise on the best model. “Best” will keep moving, and much of the work will not require it. The advantage will sit with organisations that can tell which work deserves scarce capability, which belongs on the efficiency tier, which demands sovereignty - and which needs no model at all.

Model access will be rented. The economics of consumption have to be owned.

Sources

[1] Google Cloud, Model Garden on Gemini Enterprise Agent Platform. https://cloud.google.com/model-garden

[2] Amazon Web Services, Amazon Bedrock Intelligent Prompt Routing. https://aws.amazon.com/bedrock/intelligent-prompt-routing/

[3] Microsoft Learn, Model router for Microsoft Foundry. https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-router

[4] FinOps Foundation, FinOps for AI: Tools & Services Considerations. https://www.finops.org/wg/finops-for-ai-tools-services-considerations/

[5] OpenAI, API Pricing. https://developers.openai.com/api/docs/pricing

[6] Meta, Introducing Llama 3.1. https://ai.meta.com/blog/meta-llama-3-1/

[7] Mistral AI, Introducing Mistral 3. https://mistral.ai/news/mistral-3/

[8] Microsoft Learn, Digital sovereignty. https://learn.microsoft.com/en-us/azure/azure-sovereign-clouds/digital-sovereignty

[9] Google Cloud, Sovereign Cloud. https://cloud.google.com/sovereign-cloud

[10] Microsoft Learn, AI gateway capabilities in Azure API Management. https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities

[11] Microsoft Learn, Access Foundry Models and other language models through a gateway. https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/azure-openai-gateway-guide

[12] Microsoft Learn, Deployment types for Microsoft Foundry Models. https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/deployment-types