‘Pretty Crazy’ Token Usage Is Testing Bosses’ Bet on AI


At the software company 8×8, employees are using Anthropic’s Claude to draft emails, analyze customer feedback, and write code, but so far, their growing reliance on the artificial intelligence chatbot hasn’t troubled the finance team. While other Silicon Valley companies, such as Meta, Uber, and Salesforce, have publicly expressed concerns about the growing cost of generative AI tools and have begun introducing usage caps in some cases, 8×8 says it finds itself in the black.

Over the past 18 months, the company estimates it has saved about $5 million in annual costs by canceling subscriptions to dozens of software and educational tools it deemed unnecessary in part because Claude could provide similar capabilities. So far, 8×8’s annualized bill for Claude is “well below” that figure, says Joel Neeb, the company’s chief transformation and business operations officer.

Neeb expects the savings and costs to eventually even out as 8×8 encourages more employees to adopt AI and it incorporates the tech into more complicated work. But for now, there’s still a huge gap, which “makes my chief financial officer happy,” he tells WIRED. He declined to share exact total spending on generative AI.

As companies pour hundreds of millions of dollars collectively into AI tools for coding, marketing, and customer service, a new obsession has emerged in the tech industry: “tokenomics,” or how to manage the soaring cost of AI usage. (Tokens represent the amount of content an AI model analyzes and generates.)

Last month, Royal Bank of Canada’s CEO disclosed that its token usage surged 500 percent over the past six months. At Cisco, a third of employees are using an internal AI chatbot on a daily basis, so “the token usage is getting pretty, pretty crazy,” CEO Chuck Robbins said on an earnings call. Some top engineers at analytics software developer Amplitude are “spending thousands of dollars a month or more on tokens,” according to its CEO Spenser Skates. Aaron Levine, the CEO of Box, said, “The token budgeting conversation has absolutely taken over as one of the most important” and “heated” topics.

Roughly 300 companies addressed questions or concerns about AI tokens during their earnings calls or in public discussions with financial analysts in April or May, according to a WIRED review of transcripts from the data provider AlphaStreet. That’s a small fraction of the thousands of calls held during the span, but just 93 companies mentioned “token” in April and May a year ago.

Executives at several companies said they are developing or looking to buy systems to help monitor token usage and choose the lowest-priced model for a given prompt. Others said they were still trying to figure out balancing hiring more people and increasing their budgets for tokens to achieve their goals.

Software has rarely come cheap, but the latest generation of AI tools is causing unusual stress in C-suites for a variety of reasons. Prices keep fluctuating. New models that are more powerful—and more expensive—than the last get released every month. And getting entire organizations on board with new ways of working has been a challenge, so AI-fueled productivity gains on one team can lead to bottlenecks for another.

20 Percent

That said, some companies are still encouraging employees to use AI more without worrying about the tab. In April, Long Island, New York-based clothing brand Baseball Lifestyle 101, which expects to generate $250 million in sales this year, told about 50 of its top managers to spend the equivalent of about 20 percent of their salary on AI tokens every month.

Bill Rom, cofounder and chief strategy officer of Baseball Lifestyle 101, tells WIRED the cost is likely to exceed $100,000 a month by the end of the year, but it’s already paying off. Claude recently helped land a $1 million order by identifying that a retailer was running low on some sizes of the company’s popular ice-cream-patterned shorts. “That’s a day and a half of work that can now happen in an hour or two that might make me eight figures of additional revenue over 12 months,” Rom says.



Source link

  • Related Posts

    Is the AI industry really ready to slow down?

    We’re seeing a big debate over AI safety and a potential slowdown, as Anthropic CEO Dario Amodei recently published a plan to “pace the frontier,” while Nvidia CEO Jensen Huang…

    Continue reading
    Retroid Pocket Unexpectedly Expands Its Duo Lineup With A Lite Plus Version

    The handheld maker is offering free exchanges for any Duo Lite customers who want to upgrade. Retroid Pocket If the Retroid Pocket Duo is too big and the…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    Cassidy says Trump needs to be “radically honest” on vaccines amid measles outbreak

    Cassidy says Trump needs to be “radically honest” on vaccines amid measles outbreak

    “They often don’t make it in the door”: Wardogs studio Bulkhead check for “anti-crunch messaging” on prospective hires’ social media, CEO says

    “They often don’t make it in the door”: Wardogs studio Bulkhead check for “anti-crunch messaging” on prospective hires’ social media, CEO says

    Lulu Guinness Puckers Up with a New, Surrealist Lip Design

    Lulu Guinness Puckers Up with a New, Surrealist Lip Design

    N.S. seafood processor wants accuser to name names in $10-million lobster lawsuit

    N.S. seafood processor wants accuser to name names in $10-million lobster lawsuit

    Is the AI industry really ready to slow down?

    Is the AI industry really ready to slow down?

    One-Day Cup final: Middlesex beat Leicestershire by 31 runs

    One-Day Cup final: Middlesex beat Leicestershire by 31 runs