DeepSeek’s significant price reduction of 75% on its V4-Pro model was expected to benefit enterprise AI vendors and developers. However, many are finding that lower costs do not necessarily lead to increased profit margins.
This situation arises because, while the cost of inference diminishes, agent systems are consuming tokens at a pace that exceeds the rate of price drops. For over 20 years, software economics operated under the premise that as infrastructure costs decrease, application capabilities increase. Initially, AI was thought to follow this trajectory, leading to the belief that reduced token prices would make inference a trivial expense. However, that belief is now facing serious challenges.
A typical chatbot converts one user query into a single model call, but an agent processes it through a sequence of planning, retrieval, tool usage, verification, summarization, and additional decision-making. Although the user sees a single response, the vendor incurs costs for each step, creating a significant financial burden. This introduces the “100x problem”: serving a user-initiated request through an agent’s workflow can be far more costly compared to a basic chatbot or retrieval-augmented response. For lengthy workflows, this cost can be even higher, and while model prices may decline, they do not resolve the inherent expense of a setup designed to turn one prompt into multiple operations that generate bills.
This new reality is reflected in how model providers are crafting relationships with developers. OpenAI’s initiative to allot $2 million in API credits to each Y Combinator startup signifies the rising costs associated with operating an AI company in its first year. Additionally, established firms integrating agent systems into existing products are facing even larger financial demands.
Understanding Token Amplification
In a single-interaction chatbot, one user input translates to roughly one model call, yielding an input-to-billing ratio of approximately 1:5.
However, in a multi-step agent used across various domains such as customer support or finance, this ratio can soar to 1:700 or more. Each iterative loop accumulates the ongoing conversation, tool outputs, and reasoning trails; no information is discarded.
An example query like “What did our top customer ask about last week?” usually interfaces with seven billable operations before generating a response:
-
User prompt (~50 tokens)
-
System prompt and tool definitions (~3,000 tokens, repeated per call)
-
Context retrieval (~5,000 tokens)
-
Model call #1 — tool selection (8,000 in / 200 out)
-
Tool execution (~4,000 tokens returned)
-
Model call #2 — summarization (12,000 in / 400 out)
-
Model call #3 — follow-up decision (12,400 in / 100 out)
Beginning with a single query, roughly 35,000 input tokens may be billed. Depending on the model, the cost could range from $0.10 to $0.40 per query. Multiplying this by a million monthly queries, a typical volume for any enterprise B2B feature, results in expenses reaching six figures.
The Impact on AI Business Models
The traditional pricing model for enterprise AI has revolved around seat-based SaaS: Charge a monthly fee per user for agent capabilities, which implies a manageable user cost.
However, token amplification disrupts this paradigm. A power user performing 50 agent tasks daily on a $40/month plan could incur more in inference costs than the fee itself. This contradiction damages the traditional SaaS pricing structure. As users engage more with agents, the corresponding costs can exceed what they pay, pushing vendors into negative revenue margins. Some companies now report declining gross margins for active users, revealing a direct link between AI agent usage and profit erosion.
Recent public dialogues have indicated these challenges. For instance, coverage has emerged regarding the disparity between Salesforce’s Agentforce marketing and the actual capabilities delivered. This reflects a tactical gap arising when promised features are feasible but costly to provide at the subscribed pricing. Salesforce is a notable case, albeit not the only one.
“For my team, compute costs far surpass employee expenses.” — Bryan Catanzaro, VP of Applied Deep Learning, Nvidia
The broader implication is not merely that “AI is costly”; it’s that many AI-to-business strategies are incompatible with agent-based workloads.
Illustrative Example
Take an enterprise software company charging $40 per user per month for an AI-driven support assistant. A conventional chatbot might incur negligible daily costs in inference, maintaining solid profit margins.
Replacing it with a full-fledged agent capable of ticket handling, system queries, drafting replies, validating results, and escalations might multiply the usage expense significantly. If a dedicated user runs 50 to 100 agent queries daily, inference costs could spike exponentially. What used to be a minimal infrastructure cost transforms into a substantial ongoing one.
Consequently, the users gaining the greatest benefit from the service may paradoxically contribute the least to profitability, posing an increasingly recognized challenge among enterprise solutions — the disintegration of the link between agent use and bottom-line growth.
Agent Orchestration as a Competitive Edge
Recognized strategies for managing these challenges are vital for sustainability, focusing on:
-
Cost-aware routing: Employing small classifiers to determine which processing tier manages each request can trim inference costs significantly without lowering quality.
-
Prompt caching: Major players now provide substantial discounts for cached prompts.
-
Context management: Establishing limits on tool outputs and reasoning paths can prevent unnecessary resource consumption.
-
Speculative decoding: This approach can double or triple performance efficiency on the same hardware.
“Organizations utilizing an orchestrated governance approach experience enhanced productivity, with significant impacts compared to compliance-only frameworks.” — IBM
Companies excelling in this orchestration resemble financial trading systems, with all decisions assessed for cost and profitability.
What Enterprises Should Do Next
Four strategic initiatives differentiate companies that will maintain profitability over the next two years:
-
Prioritize inference cost tracking. Monitor it across features and query types diligently.
-
Budget strategically. Set limits on costs per thousand queries and establish alerts for any overruns.
-
Integrate the router as essential infrastructure, not just an optimization. It acts as the core load balancer.
-
Quarterly prompt audits. A lengthy system prompt developed organically can evolve into a significant liability.
-
Early negotiation of volume commitments. Major vendors now offer pre-paid volume arrangements at favorable rates.
The Coming Two Years
The fundamental challenge facing agent-driven AI is not solely the expense. As indicated by DeepSeek’s pricing shift, the costs associated with frontier inference units are falling at a rapid pace.
However, the rate of token usage is outpacing these price reductions. A 75% cut in costs won’t aid a company burdened with agents performing substantially more operations per query than its pricing model can accommodate. Decisions made about architecture will now have immediate financial implications; reworking a prompt represents a chance to enhance profit margins, while a poorly designed agent loop could quickly lead to substantial costs.
Survival in the next two years will belong not to those using the cheapest models but to those whose agents are not only intelligent but also aware of their operational costs.
This presents what has become known as the 100X dilemma, emerging more rapidly than any price reductions can obscure.
Maitreyi Chatterjee is a senior software engineer at a prominent tech company.
Devansh Agarwal is an ML engineer at a leading tech organization.
Welcome to our community!
Our open guest posting program invites experts to share insights and provide in-depth analysis on AI, data management, cybersecurity, and transformative technologies shaping the future of enterprise.
Learn more about our guest post initiative, and view our guidelines if you’re interested in contributing an article yourself!
