The AI Bill Nobody Budgeted For: Why Cost Predictability Is Now a Board-Level Issue

A perspective from John Zivnuska, our COO and co-founder

Eighteen months ago, our biggest worry about AI was whether it actually worked. Today, our biggest worry is whether we can predict what it costs. That shift from "can we build it" to "can we afford it at scale" has quietly become one of the most consequential financial decisions a scaling company makes.

We're not alone in learning this the hard way. By some public accounts, Uber burned through its entire 2026 internal AI coding budget by April. Another company reportedly ran up a $500 million bill with a frontier model provider simply because no one had set usage limits. Those are extreme, enterprise-scale cautionary tales, but the underlying dynamic is exactly the one we live with every month: token-based pricing means your AI bill scales with usage, not with your revenue, your headcount plan, or your board deck. For a company our size, an unexpected 4x spike in inference spend isn't a rounding error, it's a line item that shows up in the same conversation as runway.

Frontier models are extraordinary - and priced like it

There's no disputing that frontier models from OpenAI and Anthropic are the best general-purpose reasoning engines available. But "best" comes at a specific, quantifiable price. As of this writing, OpenAI's current flagship tier (GPT-5.6 "Sol") runs $5 per million input tokens and $30 per million output tokens. Anthropic's Claude Opus 5 sits at a similar $5 / $25 per million tokens. Even Anthropic's more efficient Claude Sonnet 5, priced at $2 / $10 per million tokens, still costs several times more than the open source alternatives we'll get to below.

Multiply that by an agentic workflow that loops through dozens of tool calls, a support bot fielding thousands of daily conversations, or a batch job summarizing every customer transcript from the last quarter, and the math stops being a per-request curiosity and starts being a forecasting problem. Output tokens, the expensive side of token pricing, are generated in volume by exactly the kind of long-running, multi-step agent workflows that are becoming standard practice. Worse, that spend is variable by design: it moves with customer growth, with usage patterns you don't fully control, and with model upgrades you didn't ask for. From a COO/CFO's chair, variable cost that scales faster than revenue is the definition of a margin risk, not just an engineering budget item.

Open source gets you 80% of the way there, for a fraction of the cost

Here's what changed our thinking: the gap between frontier models and today's best open source models has narrowed dramatically for the workloads that make up the bulk of what a typical software company actually asks AI to do, including classification, extraction, drafting, internal tooling, retrieval-augmented answers, routine coding assistance, and first-pass customer support.

Run the same math on open source models and the numbers look completely different. Self-hosted or provider-hosted open models like Llama 4 Scout or Qwen3-32B run in the neighborhood of $0.80–$0.90 per million tokens on-demand and drop to roughly $0.30–$0.35 per million tokens on spot infrastructure, with the most aggressive GPU configurations pushing costs down toward $0.05 per million tokens at scale. That's roughly a 5x to 100x cost difference depending on the model and deployment path, for workloads where a well-tuned open model performs within a few points of a frontier model on the metrics that matter to our product.

For a company watching burn multiple as closely as growth rate, that's not a marginal optimization, it's the difference between AI features being a healthy gross-margin product line or a hidden cost center eating into every closed deal.

But the other 20% is exactly where frontier models earn their premium

To clarify, saying "just use open source" is as lazy a take as "just use the biggest frontier model for everything." There are workloads where the premium is worth every dollar: complex multi-step reasoning where a wrong intermediate step compounds into a wrong answer, frontier-grade coding on unfamiliar or highly complex codebases, nuanced judgment calls in customer-facing or compliance-sensitive contexts, and anything where the cost of a bad output (e.g., a lost customer, a compliance miss, a broken production deploy) dwarfs the cost of the extra, more expensive tokens. In those cases, GPT-5.6 Sol or Claude Opus 5 isn't a luxury; it's risk management priced in tokens.

The mistake we often see is defaulting every request to the frontier model because it's the path of least resistance during prototyping, then never revisiting that decision once the product ships and usage scales. That default is what turns a promising AI feature into a margin problem six months later.

Predictability is a feature, not a nice-to-have

The other half of this conversation isn't just about raw price per token - it's about variance. Frontier model pricing changes with new model releases, rate limits shift, and a single unbounded agent loop can turn a $50 experiment into a five-figure surprise on next month's invoice. Self-hosted or reserved-capacity open source infrastructure behaves more like a fixed cost: you know what a GPU-hour costs, you can cap throughput, and you can forecast spend against a roadmap instead of discovering it after the fact.

That predictability matters disproportionately as companies grow. A startup cannot absorb those model pricing volatilities the same way an established, blue-chip company can.

Our operating model going forward

The framework we've landed on, and the one we'd recommend to any founder team at our stage: default to the best open source model that clears your quality bar for a given task, and route only the workloads that genuinely require frontier-level reasoning, coding depth, or judgment to GPT-5.6 or Claude Opus/Sonnet. Instrument everything so you can see, per feature, what it costs per thousand requests and what model is serving it. Set hard usage caps and alerting before you scale a feature, not after the invoice arrives. And revisit the routing decision every quarter, because the open source frontier is moving fast enough that this month's "frontier-only" task may be next quarter's 80% solution.

AI is not going to get cheaper to ignore. It's going to get cheaper to manage well.

About the author

John Zivnuska is COO and Co-Founder of Hoonify, where he leads operations, finance, and business strategy. With more than fifteen years of experience in financial services and HPC business management, John builds the foundation that enables Hoonify's mission - making high performance computing and AI solutions accessible to everyone. He oversees company operations, legal and HR functions while managing board governance and advising on and implementing strategic growth initiatives. John earned his BSBA and MBA from the University of Colorado Boulder.