IBM CEO Pitches On-Prem AI for as Little as a Tenth of Frontier Token Cost
Krishna puts open weight models at 90 percent of frontier quality for as little as a tenth of the token cost, but the on-prem case still turns on hardware bills and scarce GPU-clustering skills.
IBM chief executive Arvind Krishna told investors on the company's Q3 earnings call that open weight models now deliver 90 percent of frontier-model effectiveness for 10 to 25 percent of the token cost, making on-premises deployment attractive for most enterprise tasks.
He framed the choice as paying ten times more for the best model or accepting the cheaper answer for tasks such as summarizing email, examining contracts and handling customer calls, which he said covers the vast majority of work. A broad benchmark might have put the top closed model at 70 and open alternatives at 20 or 30 in 2024; by late 2025 the gap had narrowed to 70 against 50.
Most businesses can build useful AI infrastructure in two or three racks, he said, but many lack the kernel, networking and GPU-clustering skills to run it, and IBM will ship software to ease deployments.
Key takeaways
- IBM's infrastructure revenue grew 5 percent year over year to $3.7 billion, slower than Dell, HPE and Lenovo. Software grew 9 percent to $7.8 billion, with Red Hat up 11 percent and annual recurring revenue up 30 percent.
- Red Hat's virtualization line, launched in 2024, has booked $800 million of deals with a $750 million pipeline. IBM links further demand to VMware users potentially on the move.
- Mainframe revenue fell 20 percent, which IBM attributes to the z17 being 18 months old. Krishna said z17 has delivered $1 billion more revenue than any previous model and that new rackable machines produced IBM Z's strongest midrange launch in nearly two decades.
- Total quarterly revenue reached $17.1 billion, up 5 percent, and the share price ended down a point or two in after-hours trading.
- Separately, a severity-one IBM Cloud incident left Classic Cloud users unable to provision resources or virtual private clouds for 90 minutes, with restoration promised by 08:00 UTC Thursday, at least 16 hours after work began. The Register asked whether critical PHP bugs announced hours earlier were the cause.
What people think
Analysis details
16 comments analyzed. Analyzed .
The sample selects active discussion branches and includes available parent comments. It may omit parts of the full thread. The analysis included 16 of 24 usable stored comments because of the input limit. Selected themes do not measure community opinion or how common a view is.
Self-hosting costs more than the API bill
A self-reported 8-GPU rig cost about $25,000 and, by its owner's own estimate, does not beat frontier-lab pricing; independence is treated as a surcharge worth paying. For some workloads a decade of Claude Code Max beats the capital outlay, and personal subscriptions at 1/40 to 1/50 of API list prices keep frontier access cheap enough to postpone switching.
Throughput and power set the real floor
Reported figures for the 8-GPU rig: roughly two seconds to first token, 100 to 150 output tokens per second, about 2,500W at full tilt and capped to 1,500W. A counterexample puts the floor lower, running Qwen Flash Next on 4xV100S plus a 48GB MacBook at 500 t/s aggregate, about 200 t/s single-user, described as roughly Sonnet 5-reliable for agentic coding.
Sovereignty is the stated draw for local inference
The stated draw of local inference is keeping work off provider systems and avoiding refusals, with the operator accepting a cost penalty for that independence. A related question asks whether renting cloud capacity can deliver equivalent governance and data-sovereignty guarantees; if it can, the on-prem case weakens.
IBM's pricing record complicates its own pitch
Selling on-prem inference invites the objection that IBM is the vendor selling the hardware. Would-be buyers say they would line up for affordable IBM kit, while the brand's pricing record draws open doubt. The counter cites IBM's PC-era pricing and the clone market, and anchors against NVidia's 95,000-euro GB300 workstation and an RTX 6000 at 20K.
Rent, own or subscribe
Renting cloud capacity is raised as an amortization play, conditional on matching governance guarantees. An arcade-to-home-console analogy casts on-prem as the eventual price-performance winner, though a multi-thousand-dollar model copy has no consumer equivalent. Self-hosting by top customers is predicted to hurt frontier labs; the counter is that rivals paying for smarter models keep the frontier valuable.
Cheap models deflate app-building value
Cheaper model access deflates the value of software work: if anyone can build the app, the idea looks worthless, and the anxiety is about jobs rather than tooling. The counter is that building an app was always roughly 1 percent of turning an idea into a business, so code was never the moat.