When Intelligence Gets Cheap: The Real AI Cost Curve Nobody's Modeling

I've sat through a dozen budget meetings this quarter where finance leaders are bracing for AI costs to spiral. The narrative is consistent: "It's getting expensive, and these subsidies can't last forever." They're modeling AI like it's a luxury SaaS contract that only ratchets up.

They're preparing for the wrong future.

This week, OpenAI and Broadcom announced their first custom chip. They called it Jalapeño — yes, really — and while the name won't win awards, the implications should terrify anyone who budgeted AI as a cost center that only grows. This isn't a product launch. It's a phase shift in who wins the next decade.

The Chip You Should Actually Care About

Here's what matters about Jalapeño: it's not for training AI models. It's for inference — the cost you pay every single time someone actually uses the model. Training is the one-time, massive upfront cost to build intelligence. Inference is the per-transaction toll every time you run it.

Broadcom taped this chip out in nine months, guided directly by OpenAI's own model roadmap. Early claims suggest performance per watt "substantially better" than today's best silicon. No full benchmarks yet, so don't crown it. But the trajectory is unmistakable.

Training stays elite and expensive. Inference is about to get industrialized.

A handful of labs will keep spending billions to train frontier models. But running those models? That's about to get cheap, power-efficient, and commoditized. Bought by the rack. Deployed everywhere.

I've Watched This Movie Before

Bitcoin mining ran this exact play, and I watched it unfold in real time advising clients navigating the crypto infrastructure boom.

It started on GPUs anyone could buy off the shelf. Then came purpose-built ASICs — chips designed for one thing and one thing only. Then immersion cooling to squeeze out waste heat. Then a global arbitrage hunt for the cheapest kilowatt-hour on earth: abandoned factories in upstate New York, geothermal plants in Iceland, flared natural gas in West Texas.

Every optimization shaved another penny off the marginal cost. The edge never belonged to whoever mined first. It belonged to whoever industrialized it cheapest.

The early miners with expensive rigs got outcompeted by operations that treated it like a manufacturing problem, not a gold rush. The romantic narrative was "digital gold." The reality was operations research and supply chain management.

AI inference is following the same arc. The question isn't whether it gets cheap. The question is whether your organization is ready when it does.

The Budget You're Building Is Already Wrong

When intelligence gets cheap to run, "who has the biggest model" stops being the question. "Who can put it to work everywhere without getting killed on cost" becomes it.

Most firms are budgeting AI as a line item that only inflates. They're modeling Moore's Law in reverse — assuming compute costs rise forever because that's what the last 18 months felt like. OpenAI burning cash on subsidized ChatGPT pricing became the mental model.

But the subsidy phase was never meant to be permanent. It was customer acquisition. The actual business model kicks in when custom silicon makes inference cheap enough to run profitably at scale.

If you're treating AI like an expensive luxury you ration carefully, you're optimizing for a world that's about to stop existing. The firms that win aren't the ones who use AI sparingly. They're the ones who wired intelligence into a thousand workflows before the cost collapsed — and then scaled without constraint when it did.

The Question Nobody Wants to Answer

Here's the uncomfortable part: which is the harder problem at your firm right now — affording the AI, or operationalizing it?

If your honest answer is the second one, the price was never your real constraint.

I see this constantly. Finance teams agonizing over AI budget allocation while the real bottleneck is change management. Legal teams debating liability frameworks while nobody's mapped which processes could actually be automated. Audit teams piloting one AI tool in isolation while competitors are rebuilding entire workflows.

The constraint isn't the technology. It's organizational willingness to use it.

The firms freaking out about cost are often the same ones who've barely deployed AI beyond a handful of pilots. They're solving for a budget problem they don't actually have yet, while ignoring the implementation problem they've had for a year.

When inference gets industrialized, that gap becomes lethal. Your competitor who integrated AI into client onboarding, document review, risk assessment, and financial analysis isn't just marginally faster. They're operating at a different cost structure entirely.

What Happens When the Constraint Disappears

Every disruption cycle has a moment when the constraint everyone was managing suddenly evaporates — and reveals the next constraint nobody prepared for.

When AWS made compute cheap, the bottleneck stopped being "can we afford servers" and became "do we have engineers who can build cloud-native architecture." Companies that spent years optimizing data center costs discovered they were solving yesterday's problem.

When electronic trading made transaction costs near-zero, the bottleneck stopped being "can we afford to trade" and became "do we have the algorithms and risk systems to trade intelligently at scale." Firms optimized for expensive, careful trades got obliterated by competitors built for volume.

When inference gets cheap, the bottleneck becomes operational maturity, not budget.

The firms that win are the ones asking different questions right now:

• Which client-facing processes could run 24/7 with AI assistance instead of during business hours with human bottlenecks?

• Where are we still using expensive human judgment on low-stakes decisions that could be automated?

• What would our service delivery model look like if intelligence cost approached zero?

Those are uncomfortable questions. They don't have clean answers. But they're the right questions for the curve we're actually on.

What to Do Monday Morning

Stop modeling AI costs as a line that only goes up. Start modeling organizational readiness as the actual constraint.

Here's what that looks like practically:

Audit your AI pilots. Which ones are still "exploring the technology" versus actually changing how work gets done? If you've been piloting for six months without production deployment, cost was never your real blocker.

Map the workflows where AI could run, not assist. Not "where could AI help a human do this faster" but "where could AI do this autonomously, with human review on exceptions only." That's the model that scales when inference gets cheap.

Ask your team: if AI inference cost dropped 90% tomorrow, what would we do differently? If the answer is "not much, we're still figuring out how to use it," you've found your real constraint.

The firms that survive technology disruption aren't the ones with the biggest budgets. They're the ones who see the curve before it bends — and position themselves on the right side of it.

The cost of AI inference is about to fall off a cliff. The cost of not being ready won't.

What's your firm's real AI constraint — budget or implementation? I'm curious whether the patterns I'm seeing hold across industries. Hit reply or find me on LinkedIn — I'd rather hear your ground truth than pretend I have all the answers.