In 2024, we invested in Together AI on a conviction: the defining companies of the next era of software would be the ones that solve the operational reality of delivering intelligence efficiently, reliably, and at production scale.

Two years later, the thesis holds: inference has become the lifeblood of the AI ecosystem, and the AI Native Cloud powering it for AI natives and enterprises, Together AI, announced their $800M Series C [NYT]. We are honored to continue partnering with CEO Vipul Ved Prakash and the Together AI team as they lead the global economy into the Inference Era.

Ahead of this year’s Montgomery Summit, we spent time with Vipul reflecting on the ways our mental model has changed over time. This is the first in a series of essays addressing those key developments and how we expect economic value to get measured and compound:

1. The token is getting cheaper, but the job is only getting bigger.
As per-token inference costs fall, the universe of tasks we ask each token to manage expands. An autonomous agent executing recursive, multi-step workloads requires several orders of magnitude more token consumption than the Q&A-style chat interface that dominated human-machine interaction the first wave of AI.

Researchers from Goldman Sachs predict a ~24x increase in token consumption by 2030 driven by agent adoption. The scale is hard to ignore: Uber rolled out Claude Code and Cursor in December and burned through its entire 2026 AI tooling budget in four months; Databricks CEO Ali Ghodsi said they have processed more than one quadrillion tokens through their AI gateway.

Token consumption has become the central cost line in AI, and while organizations treat it as a measure of productivity, “tokenmaxxing” tells us little about the outcomes or delivered intelligence.

Agentic and reasoning workloads are forcing teams to look hard at the per-token economics of every workload they run. While teams will continue getting billed in tokens in the interim, the unit that matters will ultimately be useful work delivered per dollar: did the task get done, at what cost, and with what reliability?

Company categories we’re tracking in response to this shift: small language models, agent-native runtimes and orchestration, inference-efficiency / cost-optimization layers, reliability / evaluation tooling for agentic workloads, agent observability / monitoring platforms.

2. The economic center of AI is shifting from closed to open
From our conversations with teams running production traffic, we hear that roughly 70% of LLM spend today goes to closed models leading in raw intelligence, (e.g., Fable-5, GPT-5), but the edge is narrowing. Open weights (e.g., GLM-5.2, DeepSeek, Kimi-K2, MiniMax) have achieved effective parity at one-tenth the per-token cost of closed models, and command large share of token volume.

We anticipate durable advantage will come from [i] post-training (RL, fine-tuning with proprietary data and environments), and [ii] enterprise / sovereign demand for governance and auditability. Both drivers redirect value toward open-source. Accordingly, the teams that told us they spend 70/30 in favor of closed-source today expect to land closer to 50/50 long-term.

Company categories we’re tracking in response to this shift: US-based open-weight model labs, programmatic data development, last-mile fine-tuning, and the RL/post-training environments that make open weights production-grade.

3. Inference is a three-body problem
Inference economics are set by three moving layers: the model, the silicon it runs on, and the power that caps them both. No single one can be optimized in isolation.

On the model side, new architectures show up constantly: mixture-of-experts, longer contexts, multimodality, each one reshaping the serving stack. On the hardware side, NVIDIA ships a new GPU generation every year, and a widening field of inference-specialized silicon (e.g., Google’s TPUs, Groq’s LPUs, AMD accelerators, edge NPUs) each warrant new kernels and integration work. Beneath both sits power: accelerators are increasingly power-limited and data centers are energy-constrained.

This is what separates a demo from production. A pilot runs on one model and one accelerator; production means absorbing every architecture shift, new chip, and power constraint behind a stack that must stay efficient and reliable as all three move.

4. Vertical integration and AI research DNA win in the Inference Era
This was the core thesis behind our original investment in Together AI in 2024. Inference has no point solution. When intelligence becomes a set of versatile primitives that companies shape to their own use cases, delivering it at production scale requires co-optimizing the model, silicon, and power against real workloads. These are systems-level challenges that force the research organization to become a first-class function, deeply fused with product and systems (not a side lab).

Together AI knew this from the start: Vipul Ved Prakash, Ce Zhang, Tri Dao, Chris Ré, and Percy Liang built an organization with some of the world’s best AI talent to translate AI systems research directly into full stack production advantages for delivering intelligence efficiently, reliably, and at the scale the Inference Era requires.

The open questions
Plenty has changed since 2024, and many open questions remain that we plan to address in future essays:

• How do we reconcile explosive token consumption growth with the physical limits of AI (power, hardware, networking)?
• What does it take to make an AI system reliably improve from real-world use? How do evals, monitoring, and observability feed the post-training and RL loops that make models better over time?
• As AI agents grow more capable, how much of its usefulness comes from the model vs. the infrastructure and harness around it? Where does the current agentic tech stack fall short?
• Where will value concentrate and compound as agents proliferate?

As an investment firm, we are always looking for teams across the stack building AI-native systems that convert tokens into useful work and real outcomes. If you’re building here, we’d love to hear from you as we continue this series.