‘Pretty Crazy’ Token Usage Is Testing Bosses’ Bet on AI


At the software company 8×8, employees are using Anthropic’s Claude to draft emails, analyze customer feedback, and write code, but so far, their growing reliance on the artificial intelligence chatbot hasn’t troubled the finance team. While other Silicon Valley companies, such as Meta, Uber, and Salesforce, have publicly expressed concerns about the growing cost of generative AI tools and have begun introducing usage caps in some cases, 8×8 says it finds itself in the black.

Over the past 18 months, the company estimates it has saved about $5 million in annual costs by canceling subscriptions to dozens of software and educational tools it deemed unnecessary in part because Claude could provide similar capabilities. So far, 8×8’s annualized bill for Claude is “well below” that figure, says Joel Neeb, the company’s chief transformation and business operations officer.

Neeb expects the savings and costs to eventually even out as 8×8 encourages more employees to adopt AI and it incorporates the tech into more complicated work. But for now, there’s still a huge gap, which “makes my chief financial officer happy,” he tells WIRED. He declined to share exact total spending on generative AI.

As companies pour hundreds of millions of dollars collectively into AI tools for coding, marketing, and customer service, a new obsession has emerged in the tech industry: “tokenomics,” or how to manage the soaring cost of AI usage. (Tokens represent the amount of content an AI model analyzes and generates.)

Last month, Royal Bank of Canada’s CEO disclosed that its token usage surged 500 percent over the past six months. At Cisco, a third of employees are using an internal AI chatbot on a daily basis, so “the token usage is getting pretty, pretty crazy,” CEO Chuck Robbins said on an earnings call. Some top engineers at analytics software developer Amplitude are “spending thousands of dollars a month or more on tokens,” according to its CEO Spenser Skates. Aaron Levine, the CEO of Box, said, “The token budgeting conversation has absolutely taken over as one of the most important” and “heated” topics.

Roughly 300 companies addressed questions or concerns about AI tokens during their earnings calls or in public discussions with financial analysts in April or May, according to a WIRED review of transcripts from the data provider AlphaStreet. That’s a small fraction of the thousands of calls held during the span, but just 93 companies mentioned “token” in April and May a year ago.

Executives at several companies said they are developing or looking to buy systems to help monitor token usage and choose the lowest-priced model for a given prompt. Others said they were still trying to figure out balancing hiring more people and increasing their budgets for tokens to achieve their goals.

Software has rarely come cheap, but the latest generation of AI tools is causing unusual stress in C-suites for a variety of reasons. Prices keep fluctuating. New models that are more powerful—and more expensive—than the last get released every month. And getting entire organizations on board with new ways of working has been a challenge, so AI-fueled productivity gains on one team can lead to bottlenecks for another.

20 Percent

That said, some companies are still encouraging employees to use AI more without worrying about the tab. In April, Long Island, New York-based clothing brand Baseball Lifestyle 101, which expects to generate $250 million in sales this year, told about 50 of its top managers to spend the equivalent of about 20 percent of their salary on AI tokens every month.

Bill Rom, cofounder and chief strategy officer of Baseball Lifestyle 101, tells WIRED the cost is likely to exceed $100,000 a month by the end of the year, but it’s already paying off. Claude recently helped land a $1 million order by identifying that a retailer was running low on some sizes of the company’s popular ice-cream-patterned shorts. “That’s a day and a half of work that can now happen in an hour or two that might make me eight figures of additional revenue over 12 months,” Rom says.



Source link

  • Related Posts

    Samsung S95H OLED Review: Not Quite the ‘Ultra’ Frame TV We Wanted

    Pros Punchy picture quality One of the best antireflective screens yet Bright, and excellent for gaming Cons HDR colors are inaccurate No Ambient mode despite ability to display art Every…

    SpaceX is officially buying Cursor for $60 billion

    Days after its massive IPO, SpaceX says it is spending $60 billion to buy Cursor — a bet designed to help Elon Musk’s sprawling rocket / AI / social media…

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    What to Watch in Primary and Runoff Elections in Georgia, Alabama and Oklahoma

    What to Watch in Primary and Runoff Elections in Georgia, Alabama and Oklahoma

    Jelly Roll files for divorce from wife, Bunnie XO, after nearly a decade

    Jelly Roll files for divorce from wife, Bunnie XO, after nearly a decade

    Samsung S95H OLED Review: Not Quite the ‘Ultra’ Frame TV We Wanted

    Samsung S95H OLED Review: Not Quite the ‘Ultra’ Frame TV We Wanted

    Iran Will Enter Nuclear Talks Feeling Emboldened

    Iran Will Enter Nuclear Talks Feeling Emboldened

    Judge authorizes lawsuit against coffee chains over milk alternatives

    SEO title tag Brendan Sorsby Supplemental Draft: 8 NFL teams that could target QB

    SEO title tag Brendan Sorsby Supplemental Draft: 8 NFL teams that could target QB