- AT&T processes an average of 45 billion AI tokens per day, and its Chief Data and AI Officer Andy Markus says open-weight models — which can be run on AT&T’s own infrastructure rather than rented cloud compute — are already delivering 80% to 90% cost savings in certain applications and are expected to power 70% to 80% of the company’s total AI usage over time.
- AT&T built a proprietary “smart router” that automatically directs user prompts to the most cost-effective model for each specific task, allowing it to seamlessly arbitrage across open and proprietary models based on capability requirements — a system that Markus described as the company’s defense against the “token apocalypse” of runaway AI inference costs.
- The company currently uses both open-source and open-weight models across more than a thousand internal AI applications — from summarizing customer service call transcripts to managing its core network via a customized open model called OTel, which supports AI agents that can detect root causes of network failures.
- AT&T’s commitment to open models is inseparable from what Markus calls “AI sovereignty” — the principle that enterprise data must remain under the company’s control and must not flow through vendor AI systems in ways that could expose IP. “The enterprise data is the gold mine,” he said, “and the tools are just a way to mine the gold.”
What Happened?
AT&T has made a strategic decision to build its AI future around open-weight and open-source models rather than proprietary frontier AI from vendors like OpenAI, Anthropic, or Google. Chief Data and AI Officer Andy Markus revealed that open models already account for 25% of the company’s overall AI usage and that he expects that share to reach 70-80% over time. The shift is driven by three interlocking pressures: cost (switching to open models has already cut bills by 80-90% in certain applications), data sovereignty (open models run on AT&T’s own infrastructure, keeping proprietary data fully in-house), and vendor diversification (AT&T is even experimenting with Chinese AI models to avoid becoming “beholden to any one solution”). To operationalize the strategy, AT&T built a “smart router” that automatically assigns each AI task to the cheapest model capable of handling it.
Why It Matters?
AT&T’s approach signals a potential inflection point in enterprise AI adoption. The first wave of enterprise AI use was dominated by premium proprietary models — OpenAI’s GPT series, Anthropic’s Claude, Google’s Gemini — where enterprises paid for frontier capabilities without much cost discipline. As token volumes scale to tens of billions per day, that model becomes financially unsustainable. AT&T’s playbook — smart routing, open model preference, on-premise deployment — represents the cost-optimization layer that enterprises at scale must build. Gartner data underscores how widespread this shift is becoming: the firm projects that open models will underpin more than 50% of enterprise AI use cases within two years, up from less than 10% today. That is a massive structural shift in the AI revenue landscape — from proprietary model providers toward infrastructure, fine-tuning, and deployment tooling.
What’s Next?
The AT&T model will become a template as more large enterprises hit the point where proprietary AI token costs become strategically untenable. The arms race now shifts to open model quality — Meta’s new Muse Glimmer open-weight model released this week is a direct bid for enterprise adoption — and to the tooling layer that helps companies build smart routers, fine-tuned vertical models, and on-premise deployment infrastructure. For OpenAI, Anthropic, and other proprietary AI labs, the enterprise cost-optimization trend represents a long-term competitive threat: it suggests that their revenue concentration in large enterprise accounts will erode as those customers develop the internal capability to route around premium models for most tasks, reserving frontier AI for only the highest-value, most demanding use cases.
Source: The Wall Street Journal
