AI Token Spend Deserves the Same Scrutiny as Company Headcount

Rippling's disclosure this week should be required reading for every CFO and chief AI officer. The HR software company said its spending on AI tokens had been growing about 80% a month and was on track to consume 40% of its R&D headcount budget, before it brought the bill under control. Our view is simple: AI usage now deserves the same management discipline as headcount. Not caps for their own sake, but visibility into who spends what, on which models, for which results.

What happened at Rippling?

Early this year, Rippling encouraged heavy AI use, and employees spent freely, TechCrunch reported on August 7. At a March executive meeting, the company's finance chief showed a projection that token spending was headed toward 40% of R&D headcount costs. An analysis found that roughly 10% to 15% of employees drove about 60% of AI spend, and one engineer was spending $50,000 a month. Employees also tended to default to the newest and most expensive frontier models for every task.

Rippling's response was not to ban usage. It capped spending with its main AI vendors, named "AI captains" to help colleagues use the tools well, and built its own gateway that routes each prompt to the most cost-effective model for the job. It now sells the resulting dashboards as a product called AI Spend Console, which ties spending to teams, roles and output measures such as pull requests, according to a company blog post published August 6.

The results are striking. Rippling says projected R&D token spending fell from 40% of the headcount budget to between 10% and 15%. TechCrunch reported that usage stayed high, with 600 billion tokens consumed internally in July, while July's token cost was 37% of April's. Matt MacInnis, the chief product officer, credited routing to "the more effective models."

Why token spend should be managed like headcount

Most enterprises still manage AI spend the way they managed early cloud spend: as an IT line item, approved in aggregate and reviewed after the fact. That worked when AI meant a few chat licenses. It does not work when agents and coding tools consume tokens continuously, and when a single power user can run up $50,000 in a month.

AI Connect speakers saw this coming. At COLLIDE 2024, Varun Tottempudi, Head of Data & Platforms Engineering, Global Analytics at Lockton, warned that without monitoring, auditing and alerts, "in six months you might see a bill" as large as the past five years' spending. That panel's lessons on AI ROI and FinOps across the project life cycle apply even more now that usage is spread across every employee.

Managing like headcount means three things. Spending should be attributable to a team and a purpose. It should be compared with output, not just with last month. And there should be a default path, a sanctioned model for routine work, with the expensive frontier models reserved for tasks that need them.

Is capping AI usage the wrong lesson?

There is a serious argument on the other side. Heavy experimentation is how employees discover what AI is good for. The engineers spending the most may be the ones producing the most, and capping them could slow the learning that justifies the investment in the first place. Rippling itself encouraged heavy usage early this year for much that reason.

We agree that blunt caps are the wrong tool. Rippling's own story supports that: the savings came mostly from routing and coaching, not from cutting people off, and usage stayed high. But "let everyone spend" is not a strategy either. Without measurement, a company cannot tell its most productive heavy users from its most wasteful ones, and that is the real cost of flying blind.

The fairer worry is about what gets measured. Lines of code and pull requests are crude proxies for value, and MacInnis acknowledged that the bigger challenge is measuring productivity beyond engineering. If organizations reward token-efficient activity rather than business outcomes, they will get a different kind of waste.

What should enterprise AI leaders do now?

Start with unit economics. Amey Desai, Head of AI at Nexla, closed a AI Connect panel on scalable RAG systems by urging teams to "calculate the cost per question." The same applies to agents and coding assistants: know the cost per completed task.

Next, put a gateway in front of model access so routing and policy are possible. Default routine work to smaller, cheaper models and reserve frontier models for hard problems, the approach Mistral AI's Aaron Delp described in a AI Connect webinar on right-sizing models for enterprise agents.

Finally, tie spend to outcomes the business recognizes. AI Connect panelists on moving AI from pilots to ROI found that programs stall when goals and owners are unclear. Token budgets are no different.

Rippling learned this the expensive way, and it is now selling the lesson. Other enterprises can learn it more cheaply by acting before their own finance chiefs bring a projection to the executive meeting.

Frequently asked questions

How much was Rippling spending on AI tokens?

Rippling said token spending was growing about 80% a month and was on track to equal 40% of its R&D headcount budget before it intervened.

How did Rippling cut its AI costs?

It capped vendor spending, named AI captains to coach colleagues, and built a gateway that routes each prompt to the most cost-effective model.

Should companies cap employee AI usage?

Blunt caps can slow learning. A better approach is visibility and routing: attribute spend to teams and tasks, and compare it with business output.

Opinion pieces reflect the views of The AI Enterprise Magazine Team.

Filed under: AI Strategy & ROI, AI Governance & Trust, People, Culture & Leadership