Does AI assistance enhance or erode expertise?


From a new NBER working paper:

Whether AI assistance builds or erodes professional expertise is unsettled. In a pre-registered three-month randomized controlled trial, we gave 133 practicing patent lawyers at eleven U.S. intellectual property law firms access to a custom AI drafting assistant and measured both their performance while using AI and their professional judgment afterward without it. All work was scored by blinded expert patent attorneys. Paralleling findings from other white-collar domains, AI access raised the quality of work delivered on benchmark patent drafting tasks at 10 days (0.34 SD, p = 0.03) and 90 days (0.38 SD, p = 0.01), with larger gains among junior lawyers. After three months, all subjects redlined an existing patent application without AI, a core task of patent practice requiring expert judgment. Treated lawyers outperformed controls by 0.32 SD (p = 0.04), but this advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02). Junior lawyers showed no average gain; their scores instead bifurcated, with sharply fewer mediocre scores offset by more poor and more good ones. The largest gains from AI thus accrued to the lawyers who retained the least. Foundational expertise may be a prerequisite for extracting durable skill from AI-assisted practice.

That is by David Autor, et.al.  Do note that over time the allocation of humans to tasks will evolve so that more of the humans become more productive, not less.  RCTs somehow have the odd disadvantage of requiring too many things to be held constant, and so they can miss the benefits of longer-term adjustments.




Source link

  • Related Posts

    China state newspaper blasts Anthropic’s calls to slow AI as ‘Cold War’ tactic

    BEIJING, Sept 14 (Reuters) – An essay by Anthropic’s CEO calling for a slowing down of AI development may look like a rational statement focused ‌on safety risks, but it…

    Continue reading
    Trump calls on Ukraine to halt strikes on Russian diesel fuel, citing a global shortage

    KYIV, Ukraine (AP) — U.S. President Donald Trump on Sunday called on Ukrainian President Volodymyr Zelenskyy to halt strikes on Russian oil refineries and other infrastructure used to produce and…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    Does AI assistance enhance or erode expertise?

    Does AI assistance enhance or erode expertise?

    England vs Pakistan: Joe Root and Jordan Cox come through scare to win third Test and 3-0 sweep

    England vs Pakistan: Joe Root and Jordan Cox come through scare to win third Test and 3-0 sweep

    China state newspaper blasts Anthropic’s calls to slow AI as ‘Cold War’ tactic

    China state newspaper blasts Anthropic’s calls to slow AI as ‘Cold War’ tactic

    The Last of Us Part 2 Multiplayer Mod Cancelled at Sony’s Request

    The Last of Us Part 2 Multiplayer Mod Cancelled at Sony’s Request

    Today’s NYT Connections: Sports Edition Hints and Answers for Sept. 14, #721

    Today’s NYT Connections: Sports Edition Hints and Answers for Sept. 14, #721

    How United Airlines Is Rebuilding Its 2nd-Busiest Transatlantic Hub Without Breaking It

    How United Airlines Is Rebuilding Its 2nd-Busiest Transatlantic Hub Without Breaking It